{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 房价预测案例\n",
    "\n",
    "## Step 1: 检视源数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读入数据\n",
    "\n",
    "* 一般来说源数据的index那一栏没什么用，我们可以用来作为我们pandas dataframe的index。这样之后要是检索起来也省事儿。\n",
    "\n",
    "* 有人的地方就有鄙视链。跟知乎一样。Kaggle的也是个处处呵呵的危险地带。Kaggle上默认把数据放在*input*文件夹下。所以我们没事儿写个教程什么的，也可以依据这个convention来，显得自己很有逼格。。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_df = pd.read_csv('../input/train.csv', index_col=0)\n",
    "test_df = pd.read_csv('../input/test.csv', index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 检视源数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 80 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "Id                                                                    \n",
       "1           60       RL         65.0     8450   Pave   NaN      Reg   \n",
       "2           20       RL         80.0     9600   Pave   NaN      Reg   \n",
       "3           60       RL         68.0    11250   Pave   NaN      IR1   \n",
       "4           70       RL         60.0     9550   Pave   NaN      IR1   \n",
       "5           60       RL         84.0    14260   Pave   NaN      IR1   \n",
       "\n",
       "   LandContour Utilities LotConfig    ...     PoolArea PoolQC Fence  \\\n",
       "Id                                    ...                             \n",
       "1          Lvl    AllPub    Inside    ...            0    NaN   NaN   \n",
       "2          Lvl    AllPub       FR2    ...            0    NaN   NaN   \n",
       "3          Lvl    AllPub    Inside    ...            0    NaN   NaN   \n",
       "4          Lvl    AllPub    Corner    ...            0    NaN   NaN   \n",
       "5          Lvl    AllPub       FR2    ...            0    NaN   NaN   \n",
       "\n",
       "   MiscFeature MiscVal MoSold  YrSold  SaleType  SaleCondition  SalePrice  \n",
       "Id                                                                         \n",
       "1          NaN       0      2    2008        WD         Normal     208500  \n",
       "2          NaN       0      5    2007        WD         Normal     181500  \n",
       "3          NaN       0      9    2008        WD         Normal     223500  \n",
       "4          NaN       0      2    2006        WD        Abnorml     140000  \n",
       "5          NaN       0     12    2008        WD         Normal     250000  \n",
       "\n",
       "[5 rows x 80 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这时候大概心里可以有数，哪些地方需要人为的处理一下，以做到源数据更加好被process。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: 合并数据\n",
    "\n",
    "这么做主要是为了用DF进行数据预处理的时候更加方便。等所有的需要的预处理进行完之后，我们再把他们分隔开。\n",
    "\n",
    "首先，SalePrice作为我们的训练目标，只会出现在训练集中，不会在测试集中（要不然你测试什么？）。所以，我们先把*SalePrice*这一列给拿出来，不让它碍事儿。\n",
    "\n",
    "我们先看一下*SalePrice*长什么样纸："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[<matplotlib.axes._subplots.AxesSubplot object at 0x10864a5f8>,\n",
       "        <matplotlib.axes._subplots.AxesSubplot object at 0x1092429b0>]], dtype=object)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnX24HVV97z/fEIhEMCe8nYMJcBQVsFUjKkmtVw6vErwF\n2iuCVuUgvVKRKnqrRGtLaa0C97EGahXaIjf6VEHwBVQkaMmxtiqCZCtKCAFySELI4TUgbxHI7/6x\n1k7mbPY+e/Y5M7Nnzv59nmc/e2bNmu+sWbP2/GbWd81smRmO4ziOU2dGtwvgOI7jlAsPDI7jOM44\nPDA4juM44/DA4DiO44zDA4PjOI4zDg8MjuM4zjh6NjBIWivp8Iy0jpb0zSms/05J12VRlryR9A1J\nR3e7HM70QdKbJK3qdjmc7ahXn2OQtBY4zcxuyEDrJuAMM7tp6iXrLpIGgEuA1wN7A4Nmti6x/A3A\nF83s9V0qouM4OdOzdwxZIen1wIsmGxQk7ZBxkVptZ62kfVNk3Qp8H/gT4HlXDXE/d5V0cMZFdHqQ\notq/0xkeGABJO0laKuleSRskfU7SjonlH5O0MS47TdJWSS+NixcDP2rQ2yrpLyTdJel+SRcklp0i\n6b8k/aOkh4BzYtqPE3l+T9L1kh6SdJ+kJTFdkpZIulPSA5Iul9SXcjdT3Rqa2f1mdjFwM6AW2X4E\nvDXldp0eJF6ILJH0m9iOL42/s0MlrY+/qfuAL9XTEuvOj12W98d2flFi2Xsl3RY1v5/yYsfpEA8M\ngU8ChwCvBl4Tpz8JIOkY4CzgcOBlwKGMP8m+CljdRPME4OD4OV7SexPLFgJ3AnsC/xDTLG5vF+AH\nwLWErpyXAf8R83wIOA74H8CLgUeAL6Tcx1Yn+cmwilBPjjMR7wSOAvYHDiD+poABoA/YF3hfTKu3\n/xnAd4G1cfk84PK47ARgCeG3tSfwY+BrBexHz+GBIfBO4Fwze8jMHgLOBd4dl50IXGZmt5vZ03FZ\n8iTbB/y2ieZ5ZvaomW0AlgLvSCy718y+YGZbzWxLw3r/E7jPzJaa2e/M7IlEN9X7gL8ys/vM7Bng\n74C3xR9TGrIKDr8l7LfjTMQ/mdlGM9tMuACq/waeA84xs2eatP+FhAuij5nZ0/E38JO47H3AZ8zs\nDjPbCpwHLJC0TwH70lN4YAi8GFiXmL8nptWXrU8sS05DuGrftYnmhhZ6zTSS7APc1WLZfsC3JD0s\n6WHgNuAZoL8xo6R9JD0S8z5CuPr6ZSLt5AnK0I5dgc1TWN/pDVr9Bh6IFzbNmA/cE0/8jewHXJho\n/w8R7jTmZVVgJzCz2wUoCRsJja4+ZG6/mAZwH6Gx1tmX8V1JvwJe0URzn4Tevgk9mLi/fz3j7y6S\nrAPea2Y/nWD9sAGz9cDc+ryku4FDY/pUOQj4ZQY6zvQmeSWf/E21a//7SprRJDisAz5lZt59lDN+\nxxD4GvBJSXtI2gP4a+ArcdnXgVMlHShpdlyW5FpgqInmRyX1xdvcDxH7SVPwXaBf0gejWbeLpEPi\nskuAT9cNN0l7Sjoupa5I2ZUkaRbwgjj7gjif5FDCyCXHmYgPSJonaTfg42z/DUzUDn9OuBg7T9Js\nSbMkvTEuuwT4hKRXAkiaI+lteRW+l+nlwJC8avkUYRTOrwhXwjcTTWEzuw64CFgB3AHU+zu3xOUr\ngc1xfH+Sq4FfALcA3wG+lKpQZo8TDLvjgE1xm0Nx8YVR93pJj8ayHNJEpql0ynwATwGPxXVuB56s\nL4j7+biZ3dyBntObfBW4njDQ4k4aBlo0I94l/BHwcsIdwnrg7XHZtwm+wuWSNhN+r8fkVfheJtUD\nbpJGgUcJY9yfMbNDJM0FriDcIo4CbzezR2P+iwjDOJ8Ahs2slkvpu4CkA4FbgVn1W11JRwHvN7M/\nifNbgZeZ2d3dK2k+SLoK+FczW97tsuSJpA8DpxHa/K3AqYQ+8ssJXXS3AO82s2cl7QR8GXgd8CBw\nUvKhwF4kywdIneJJe8ewFRgys9eaWf0KdQnwQzM7ALiBcKuIpMXA/mb2cuB04OKMy1w4kk6QtGMM\nhucD1yT7P83sB/WgMN0xs7f1QFB4MfAXwMFm9mqCF/cOwrH/bGzzmwmBg/j9cGzzS4ELnq/qONWh\nk2GOjXmPB5bF6WVxvp7+ZQAzuxGYI+l5o2YqxunAA8AawiigM9rk7833jEwvdgBeKGkmsDPBOD0M\n+EZcvowwnh7G/xauAo4osJxlxX8DFSbtqCQDlksy4BIz+zeg38zGAMxsk6S9Yt55jB+OeW9MG8uo\nzIVjZos7zO+P+VcYM9so6bOEPu4nCf3ktwCbE3eKG9g+THJbmzez5yRtlrSbmT1ccNFLg5m9tH0u\np6ykDQxvjCf/PQnG52paXxE0G3HgVw9OZYivGTme4J89ClxJ8MwaqbfrxjYvvM07FSZVYDCzTfH7\nAUnfJoyEGZPUb2ZjCm/kvD9m38D48cvzGT+GH4B49+E4U8bMsnzdB8CRwN31K35J3wLeCPQlxtcn\n23W9zW9UeCnci8zskUZRb/NOVuTQ5sfR1mOIY4l3idMvBI4mjNK4BhiO2YYJwyiJ6e+J+RcRbr+b\ndiOZ2ZQ/55xzTql0ylim6apjltt5dh2wSNILJIngGfyGMGT5xJjnlIY2f0qcPpEwGKMp3sZ6UydL\nrSJIc8fQT3gNg8X8/25m10u6Gfh6fDncOuIPxsyulXSspDsJw1VPzansAIyOjpZKJ0st1+kOZvbz\nOCx3JWGwwUrgXwgPM14u6e9j2qVxlUuBr0haQ3hNw1ReN5KKsh0L1ylWK2/aBgYzWwssaJL+MOGW\nu9k6Z069aI7TPczsXMILE5OsJbzkrTHvFuJDWI4zHaj8k8/Dw8Ol0slSy3WcVpTtWLhOsVp507W/\n9pRk3dq2M32QhOVsxGWFt3knC4po85W/YxgZGSmVTpZaruO0omzHwnWK1cqbygcGx3EcJ1u8K8mp\nNN6V5PQa3pXkOI7jFE7lA8N07kt0HacVZTsWrlOsVt5UPjA4juM42eIeg1Np3GNweg33GKYhAwOD\nSEr1GRgY7HZxHcfpQSofGKrWlzg2dg/hjcztPyFvvuWpsk4vU7Zj4TrFauVN5QOD4ziOky3uMRRM\neItz2v1WYa/ZrSruMTi9hnsMjuM4TuFUPjB4X2J7ylZHZaufKlK2Y+E6xWrlTeUDg+M4jpMt7jEU\njHsM2eIeg9NruMfgOE5LOnkmxp+NcTqh8oHB+xLbU7Y6Klv9VJGRkZGOnolp9WxM2Y7pdNXJWitv\nKh8YHCdrJL1C0kpJt8TvRyV9UNJcSddLWi1puaQ5iXUukrRGUk3S8/4j3XGqhHsMBeMeQ7bk3d8q\naQawAVgInAk8ZGYXSDobmGtmSyQtBs40s7dKWghcaGaLmmhl2uY7a0vb1vI2VXHcY3Cc7nMkcJeZ\nrQeOB5bF9GVxnvj9ZQAzuxGYI6m/6II6TlZUPjB4X2J7ylZHZaufNpwEfDVO95vZGICZbQL2iunz\ngPWJde6NablRtmPhOsVq5U3lA4Pj5IWkHYHjgCtjUqs+mGa39d5f41QW9xgKxj2GbMmzv1XSccAZ\nZnZMnF8FDJnZmKQBYIWZHSTp4jh9Rcx3O3Bo/e4ioWennHIKg4ODAPT19bFgwQKGhoaA7VeUaedD\nW1oBDMUtjMTvieYP29amOt2ez3dnvj49OjoKwLJly3L3GDwwFIwHhmzJOTB8DbjOzJbF+fOBh83s\nfElLgL5oPh8LfCCaz4uApW4+O3nh5nMKvC+xPWWro7LVTzMk7Uwwnr+ZSD4fOErSauAI4DwAM7sW\nWCvpTuAS4Iy8y1e2Y+E6xWrlzcxuF8BxyoiZPQXs2ZD2MCFYNMt/ZhHlcpwi8K6kgvGupGzp5Xcl\neVdSb+JdSY7jOE7hVD4weF9ie8pWR2WrnypStmPhOsVq5U3lA4PjOI6TLe4xFIx7DNniHoN7DL2G\newyO4zhO4VQ+MHhfYnvKVkdlq58qUrZj4TrFauVN5QOD4ziOky2pPYb4XvqbgQ1mdpykQeByYC5w\nC/BuM3tW0k6EVxC/DngQOMnM1jXRc4+hfW7vD26DewzuMfQaZfMYPgTclpg/H/ismR0AbAZOi+mn\nEd4n83JgKXBBFgV1HMdxiiFVYJA0HzgW+LdE8uHAN+L0MuCEOJ38M5OrCO+UyQ3vS2xP2eqobPVT\nRcp2LFynWK28SXvH8Dngo8T7Vkm7A4+Y2da4fAPb/5hk25+WmNlzwGZJu2VWYsdxHCdX2noMkt4K\nLDazMyUNAR8B3gv8NHYX1e8ovmdmr5H0a+BoM9sYl90JvMHMHmnQzfTd9FWZP+ywwwjxdSTWxFD8\nbjZ/NPAMaZg7t59vfvPyru/fdHw3fVa4x+BkQREeQ5rA8GngXcCzwM7ArsC3CWetATPbGt9Bf46Z\nLZZ0XZy+UdIOwH1mtlcTXTef2+fuKG+v1qcHho7W6sl2Mp0ohflsZp8ws33N7KXAycANZvYuwl9H\nnRiznQJcHaevifPE5TdkW+TxeF9ie8pWR2WrnypStmPhOsVq5c1UnmNYAnxE0h3AbsClMf1SYA9J\na4CzYj7HcRynIvi7kgrGu5KyxbuSvCup1yhFV5LjOI7TW1Q+MHhfYnvKVkdlq58qUrZj4TrFauVN\n5QOD4+SBpDmSrpS0StJvJC2UNFfS9ZJWS1ouaU4i/0WS1kiqSVrQzbI7zlRxj6Fg3GPIlrz6WyX9\nP+BHZnaZpJnAC4FPAA+Z2QWSzgbmmtkSSYuBM83srZIWAhea2aImmu4xOFOmFM8x5LZhDwxpcneU\nt1frM+sfiaRdgZqZ7d+QfjtwqJmNSRoAVpjZQZIujtNXxHyrgCEzG2tY3wODM2XcfE6B9yW2p2x1\nVLb6acJLgQclXSbpFkn/Imk20F8/2ZvZJqD+4Oa218BE7mX7K2JyoWzHwnWK1cqbmd0ugOOUkJnA\nwcAHzOxmSZ8jPI/T6lK72dVb07zDw8OZvQYmMMLEr1VpNs82vVqt1vXXnCTnp3N5arXapNavT9df\nA1ME3pVUMN6VlC05dSX1E94F9tI4/yZCYNif2EXUpitpW5dTg653JTlTxruSHKcLxBP6ekmviElH\nAL8hvO5lOKYNM/41MO8BiO8N29wYFBynSlQ+MHhfYnvKVkdlq58WfBD4d0k14DXApwl/TnWUpNWE\nYHEegJldC6yNbxK+BDgj78KV7Vi4TrFaeeMeg+M0wcx+CbyhyaIjW+Q/M98SOU5xuMdQMO4xZIu/\nK8k9hl7DPQbHcRyncCofGLwvsT1lq6Oy1U8VKduxcJ1itfKm8oHBcRzHyRb3GArGPYZscY/BPYZe\nwz0Gx3Ecp3AqHxi8L7E9ZaujstVPFSnbsXCdYrXypvKBwXEcx8kW9xgKxj2GbHGPwT2GXsM9Bsdx\nHKdwKh8YvC+xPWWro7LVTxUp27FwnWK18qbygcFxHMfJFvcYCsY9hmxxj8E9hl7DPQbHcRyncCof\nGLwvsT1lq6Oy1U8VKduxcJ1itfKm8oHBcRzHyRb3GArGPYZscY/BPYZewz0Gx3Ecp3AqHxi8L7E9\nZaujstVPFSnbsXCdYrXypvKBwXHyQNKopF9KWinp5zFtrqTrJa2WtFzSnET+iyStkVSTtKB7JXec\nqeMeQ8G4x5AtefW3SrobeJ2ZPZJIOx94yMwukHQ2MNfMlkhaDJxpZm+VtBC40MwWNdF0j8GZMu4x\nOE73EM//fRwPLIvTy+J8Pf3LAGZ2IzBHUn8RhXScPKh8YPC+xPaUrY7KVj8tMGC5pJsk/VlM6zez\nMQAz2wTsFdPnAesT694b03KjbMfCdYrVypuZ3S6A45SUN5rZJkl7AtdLWk3rfptmt/VN8w4PDzM4\nOAhAX18fCxYsYGhoCNh+4kg7HxgBhhLTpJhnm16tVpv09vOYn87lqdVqk1q/Pj06OkpRtPUYJM0C\n/hPYiRBIrjKzcyUNApcDc4FbgHeb2bOSdiLcVr8OeBA4yczWNdF1j6F97o7y9mp95t3fKukc4HHg\nz4AhMxuTNACsMLODJF0cp6+I+W8HDq3fXSR03GNwpkwpPAYz2wIcZmavBRYAi6PBdj7wWTM7ANgM\nnBZXOQ142MxeDiwFLsil5I6TE5JmS9olTr8QOBq4FbgGGI7ZhoGr4/Q1wHti/kXA5sag4DhVIpXH\nYGZPxslZhLsGAw4DvhHTlwEnxOmkQXcVcEQmJW2B9yW2p2x1VLb6aUI/8F+SVgI/A75jZtcTLoaO\nit1KRwDnAZjZtcBaSXcClwBn5F3Ash0L1ylWK29SeQySZgC/APYH/hm4i3BVtDVm2cB2s22bEWdm\nz0naLGk3M3s405I7Tk6Y2VrC3XFj+sPAkS3WOTPvcjlOUXT0HIOkFwHfAs4BvmRmr4jp84Hvmdlr\nJP0aONrMNsZldwJvSI4Hj+nuMbTP3VHeXq1Pf1dSR2v1ZDuZThTR5jsalWRmj0n6EbAI6JM0I941\nzAc2xmwbgH2AjZJ2AF7UGBTqZDlCoyrz26nPD7WZT5s/bKPb+1dE/RU9QsNxeg4zm/AD7AHMidM7\nE0YoHQtcQRhxBPBF4M/j9BnAF+L0ycDlLXQtC1asWFEqnXZagIGl/HSWN+99K5uO2bb9btuOy/DJ\nqs2bhTrsrH00bydlO6bTVSdLrSLafJo7hr2BZdFnmAFcYWbXSloFXC7p74GVwKUx/6XAVyStAR6K\nwcFxHMepCP6upIJxjyFb3GNwj6HXKMVzDI7jOE5vUfnA4OOV21O2Oipb/VSRsh0L1ylWK28qHxgc\nx3GcbHGPoWDcY8gW9xjcY+g13GNwHMdxCqfygcH7EttTtjoqW/1UkbIdC9cpVitvKh8YHMdxnGxx\nj6Fg3GPIFvcY3GPoNdxjcBzHcQqn8oHB+xLbU7Y6Klv9VJGyHQvXKVYrbyofGBzHcZxscY+hYNxj\nyBb3GNxj6DXcY3Acx3EKp/KBwfsS21O2Oipb/VSRsh0L1ylWK28qHxgcJy8kzZB0i6Rr4vygpJ9J\nWi3pa5JmxvSdJF0uaY2kn0rat7sld5yp4R5DwbjHkC159rdK+jDwOsLf0x4n6QrgKjO7UtIXgZqZ\nXSLp/cCrzOwMSScBf2xmz/uDKvcYnCxwj8FxuoSk+YS/sP23RPLhwDfi9DLghDh9fJwHuAo4oogy\nOk5eVD4weF9ie8pWR2WrnxZ8Dvgo8ZJc0u7AI2a2NS7fAMyL0/OA9QBm9hywWdJueRaubMfCdYrV\nyps0//nsOD2FpLcCY2ZWkzRUT46fJJZYNk6CFn08w8PDDA4OAtDX18eCBQsYGgqbqJ840s4HRoCh\nxDQp5tmmV6vVJr39POanc3lqtdqk1q9Pj46OUhTuMRSMewzZkkd/q6RPA+8CngV2BnYFvg0cDQyY\n2VZJi4BzzGyxpOvi9I2SdgDuM7O9mui6x+BMGfcYHKcLmNknzGxfM3spcDJwg5m9C1gBnBiznQJc\nHaevifPE5TcUWV7HyZrKBwbvS2xP2eqobPXTAUuAj0i6A9gNuDSmXwrsIWkNcFbMlytlOxauU6xW\n3rjH4DgTYGY/An4Up9cCC5vk2QK8veCiOU5uuMdQMO4xZIu/K8k9hl7DPQbHcRyncCofGLwvsT1l\nq6Oy1U8VKduxcJ1itfLGPQbH6SlmxS6o9PT378emTaP5FMcpJe4xFIx7DNniHkPnHoP7EtXGPQbH\ncRyncCofGLwvsT1lq6Oy1U8Vya4Os9EpW9som07WWnlT+cDgOI7jZIt7DAXjHkO2uMfgHkOv4R6D\n0wFhtEmaz8DAYLcL6zhOial8YPC+xDpbCFeCzT4rxs2Pjd0zqS2Usa57FfcYqqWTtVbeVD4wOI7j\nONniHkPB5Okx9KIf4R6Dewy9Rik8BknzJd0g6TZJt0r6YEyfK+l6SaslLZc0J7HORZLWSKpJWpDn\nDjiO4zjZkqYr6VngI2b2SuAPgA9IOpDwzvkfmtkBhD8m+TiApMXA/mb2cuB04OJcSh7xvsQ0jGSj\nUsK67lXcY6iWTtZaedM2MJjZJjOrxenHgVXAfOB4YFnMtizOE7+/HPPfCMyR1J9xuR3HcZyc6Mhj\nkDRIuMT4fWC9mc1NLHvIzHaX9B3gM2b2k5j+Q+BjZnZLg5Z7DO1z55Z3utS9ewzuMfQapfAYEoXZ\nBbgK+FC8c2jVUpoV2FuV4zhORUj12m1JMwlB4StmVv8D9DFJ/WY2JmkAuD+mbwD2Saw+H9jYTHd4\neJjBwUEA+vr6WLBgAUNDQ8D2/rh28/W0tPlbzS9dunRS228231i2xvIG6vNDbebT5q+nNVue1Hp+\nedPO12o1zjrrrEmvX5+fqH7SHO+RkRFGR0fpZbL1GIamrjIysu1YuU7+WrljZm0/BM/gHxvSzgfO\njtNLgPPi9LHA9+L0IuBnLTQtC1asWFEqnXZagIGl/GSVd8Xz8ma9X93QMdu2L6nacdoPMAu4EVgJ\n3AqcE9MHgZ8Bq4GvATNj+k7A5cAa4KfAvi10M9vvFStWdNg+WrWTxrbRfJ005clqv6ajTpZaebT5\nxk9bj0HSHwL/GX8g9cdnPwH8HPg64e5gHXCimW2O63weOAZ4AjjVGvyFmMfabXs64h5DtuTV3ypp\ntpk9KWkH4L+BDwEfAa4ysyslfRGomdklkt4PvMrMzpB0EvDHZnZyE81M27x7DL1JER6DP+BWMB4Y\nsiXvH4mk2YQLozOA7wIDZrZV0iLCncRiSdfF6RtjINlkZns20fLA4EyZUpnPZcXHK6dhJBuVEtZ1\nXkiaIWklsAn4AXAXsNnMtsYsG4B5cXoesB7AzJ4DNkvaLc/y+XMM1dLJWitv/D+fHacJMQC8VtKL\ngG8BBzXLFr8br95aXpZnMeBivIE5QvsBDI3zyXVrqfNPdYBH2gEOeep3szy1Wm1S69enixxw4V1J\nBeNdSdlSSH+r9DfAk8DHSNeVdJ+Z7dVEx7uSnCnjXUkVYGBgMPX/IIQfslN2JO1Rf/eXpJ2BI4Hb\nCO8vPzFmOwWoD92+Js4Tl99QXGkdJ3sqHxi63ZcY/tvAGj4rmqTVP91gJBuVEvbb5sTewApJNcKw\n1eVmdi1hWPZHJN0B7AZcGvNfCuwhaQ1wVsyXK+4xVEsna628cY/BcRows1uBg5ukrwUWNknfAry9\ngKI5TiG4xzBFOu/ndY8hS/xdSe4x9BruMTiO4ziFU/nAUMa+xKz6bbNjJBuVUtZ1b+IeQ7V0stbK\nm8oHBsdxHCdb3GOYIu4xdBf3GNxj6DXcY3Acx3EKp/KBoYx9ie4xFKPTy7jHUC2drLXypvKBwXEc\nx8kW9ximiHsM3cU9BvcYeg33GBzHcZzCqXxgKGNfonsMxej0Mu4xVEsna628qXxgcBzHcbLFPYYp\n4h5Dd3GPwT2GXsM9BsdxHKdwKh8YytiX6B5DMTq9jHsM1dLJWitvKh8YHMdxnGxxj2GKuMfQXdxj\ncI+h13CPwXEcxymcygeGMvYlusdQjE4v4x5DtXSy1sqbygcGx8kaSfMl3SDpNkm3SvpgTJ8r6XpJ\nqyUtlzQnsc5FktZIqkla0L3SO87UcY9hirjH0F3y6G+VNAAMmFlN0i7AL4DjgVOBh8zsAklnA3PN\nbImkxcCZZvZWSQuBC81sURNd9xicKeMeg+N0ATPbZGa1OP04sAqYTwgOy2K2ZXGe+P3lmP9GYI6k\n/kIL7TgZUvnAUMa+RPcYitEpAkmDwALgZ0C/mY1BCB7AXjHbPGB9YrV7Y1puuMdQLZ2stfJmZrcL\n4DhlJXYjXQV8yMwel9SqP6XZbX3TvMPDwwwODgLQ19fHggULGBoaArafONLOB0aAocQ0KeaT69ZS\n5++0fJOZr9Vquep3szy1Wm1S69enR0dHKQr3GKaIewzdJa/+Vkkzge8C3zezC2PaKmDIzMaiD7HC\nzA6SdHGcviLmux04tH53kdB0j8GZMu4xOE73+BJwWz0oRK4BhuP0MHB1Iv09AJIWAZsbg4LjVInK\nB4Yy9iW6x1CMTl5I+kPgT4HDJa2UdIukY4DzgaMkrQaOAM4DMLNrgbWS7gQuAc7Iu4zuMVRLJ2ut\nvHGPwXEaMLP/BnZosfjIFuucmV+JHKdY3GOYIu4xdBd/V5J7DL2GewyO4zhO4bQNDJIulTQm6VeJ\ntNK8GqCMfYnuMRSj08u4x1Atnay18ibNHcNlwFsa0pYAPzSzA4AbgI8DxFcD7G9mLwdOBy7OsKyO\n4zhOAaTyGCTtB3zHzF4d57eN024znnvbuO8mmu4xdDHvdKh7cI/BPYbeo8wew15leTWA4ziOky1Z\nD1dN/WoAyOb1APW0qT6uvnTp0klvP1CfH2J8v+1Qw/Jm+SeaT5u/ntZs+fPLM9nXA5x11lmTXr8+\n33jsOlm/Pl3k6wHKSLYew9DUVUZGGl7V4Tp5auWOmbX9APsBv0rMryK8UAxgAFgVpy8GTkrku72e\nr4mmZcGKFSu6qgMYWMNnRZO0+qdZ/rzzNpZncnXf7bpuRtyXVO2425+s2rxZqMPO2kerdjJRW03f\nXsrWNsqmk6VWEW0+rccwSPAYXhXnzwceNrPzJS0B+iy8l/5Y4AMW3ku/CFhqTd5LHzUszbbLjnsM\n3cU9BvcYeo1SeAySvgr8BHiFpHWSTiW8CqAUrwZwnKrz5jcfy4wZMzr67LTTzt0utjONaRsYzOyd\nZvZiM5tlZvua2WVm9oiZHWlmB5jZUWa2OZH/TDN7mZm9xsxuybf45Ryv7M8xFKMzXbjrrrWY3YrZ\ns6k/O+10YEZbH8lGpWRto2w6WWvljT/53JPMQlKqz8DAYLcL2yPM6PBTid4zp6L4u5KmSFU9huni\nR0wHj2HevIPYuPGbwEGptebMOYJHH70B9xh6j1J4DI7j9Drp7zD9TnN6UPnAUMa+RPcYitHpZZ59\n9pGMlEZS5NlCuMuY6LNi3PzY2D2TK03J2ph7DI7jOI6DewxTxj2G7uIeQzEeg/sS5cE9BsdxHKdw\nKh8YytiX6B5DMTp5Uvb/ISnWYyhOp2xtzD0Gx3GSXIb/D4nTo7jHMEXcY+guefa3Zv0/JO4xOFng\nHoPjlAtw46IfAAAN6ElEQVT/HxKnJ6h8YChjX6J7DMXolIiO/ockC9xjqJZO1lp5k/Uf9TjOdGZM\nUn+iK+n+mL4B2CeRbz6wsZlAsz+n2s5I/B5KOV9PS5u/Pp9ct9Zh/vTlm+yfQU32z7bymM+yPLVa\nbVLr16eL/HMq9ximiHsM3SVnj2GQDP+HxD0GJwuK8Bj8jsFxmhD/h2QI2F3SOuAcwv+OXCnpvcA6\n4EQI/0Mi6dj4PyRPAKd2p9SOkw3uMWSsE9Uy1MqCkWxUSlnX+VD2/yFxj6FaOllr5U3lA0MeDAwM\npn6LpOM4znTDPYYmdOYbuMfQTfxdSe4x9Br+HIPjOI5TOJUPDGXsS3SPoRidXsY9hmrpZK2VN5UP\nDI7jOE62uMfQBPcYxuct63EC9xjcY+g93GNwHMdxCqfygaGMfYnuMRSj08u4x1Atnay18qbygcFx\nHMfJFvcYmuAew/i8ZT1O4B6Dewy9h3sMjuNUlFmp3x5Q/wwMDHa70E6k8oGhjH2J7jEUo9PLlN9j\n2EK4y0j/GRu7p3RtzD0Gx3Ecx8E9hqa4xzA+b1mPE7jHUGaPwX2JfHCPwXEcxymcygeGMvYlTi+P\noTMTMY2BWKW+1rJSfo9hkiol+z27x+A4TUmaiCtIYyA6jlNt3GNognsMk80b8hd5XN1jKK9f4B5D\nPrjHkCH+r2yOU3b82YeykEtgkHSMpNsl3SHp7Dy2USdtv13o4pioGyTZTTLlUmWgkSUjpdKpUl9r\nJxTZ7qenx7CFNN2Vabou3WOYGpkHBkkzgM8DbwF+D3iHpAOz3k6dWq2WlVJGOllrZUG56ii7Y1Ye\nim73zz3324yUytU2ytbGsmyrVWr3edwxHAKsMbN7zOwZ4HLg+Kw38pd/+TfMnj2Xj350CbNnz237\nac/mDEuXpVYWZFWeNDrtuwM+/OEPT8dugELafR2zZzNSKrJtFKezeXO5dLLWypuZOWjOA9Yn5jcQ\nfjSZUqv9hqeeWgrcxrPPLmmT+/8Cn8m6CE5T6qOYJuJvgb9lbGxa+TmFtHvHKYI8AkOzX3vmQw1e\n8IIdmT37n/jd7zYxe/ZtE+bdsuUOtmxppziaVdEy1sqC0WmqUyom3e5nzdqRXXZ5PzNm7Jp6Y48/\nvjF13okZrbjOrJYDRs4999ym6TNmzGbr1idTb+Hcc8/teJ1W22pVpiT9/fuxadNox9vKksyHq0pa\nBPytmR0T55cAZmbnN+TzcWlOJpRhuGqadu9t3smKvNt8HoFhB2A1cARwH/Bz4B1mtirTDTlOifB2\n70wnMu9KMrPnJJ0JXE8wty/1H4cz3fF270wnuvbks+M4jlNSzCzTD3ApMAb8KpE2l3AltRpYDsxp\nse4pwB0x34+moPMccAuwErinic7bgF/HfAdPsC/HALfHMp3dYt/Sao0Cv4xleqCJzgXAKsJA7m8A\nL0pRppumoNOuPH+XWH4dMDDJY5ZWJ3nMvt2srhN5/xLYCuyWokzvybqNT/J3UT9ujwK/TduugYuA\nNfF4LkiknwU8SRgGth74YEw/NOr/Lh7jOYlt3BPTHwPeFNNnAXcBT0etb8b0QeBXMf0x4MKYvlNs\nV08AT8VjXt/Gx2NZnwJ+mtBZFbUfAz6e0PlOzPsUcDUwMy57JOZ/Grg9Uf5fxPQngHMSdXFZ1Hga\n+H5C589jXWwhjBJbGHV+GvM+AXwhoXNJTHsa+Elivz4W056Odfoo8MFO6zou+2pi334S62Eqdb0K\nOLrVOSuRPgj8jNDGvlavo5btNYcfwJuABYxv+OcDH4vTZwPnNVlvLqGBzgH6gHujVkc6cdljbcpz\nAPBy4AZanMwJ3QF3AvsBOxJ+mO+cjFbMdzcwd4IyHQnMiNPnAZ9JUaY1wAmd6qQszy6J6b8AvjjJ\nY9ZWp/GYtSpTTJ9PCDBraRIYmpTpLlpcQBT1aThuh8Yf5+p27RpYDHwvTi8EfpbYx9FYR33xWK4B\nDgQ2Av8U890OXBGnvw6sitOfB+5NaN0d62s3wsnpcOCKqHkI8EXgVsLDe+8nBPCPAScRfhfnAa8k\nBPX/A3yLcDJT1Lkv7vslhCB2YNRZA5wYddYAp0edLcAehJPZnVHnfODBqPNxwkXDgbGONkadhTH9\n9LhfjwFnJNrBPlFnXdyvs2P6W6LOXXG/FsY857VoT/dFrU7r+iDgmbhvfcDjhOA12bqe2VBHzc5Z\nB8ZtXwGcGKe/CJw+YZvN6YewH+NPDrcD/XF6gHgV0LDOySROGrHwH+hUJy777UTlSaSvoHVgWAR8\nPzG/JDakjrXi8rXA7u3KFJedAHwlZZk+06nOJMqzBPjnyRyzNDrNjlmrMgFXAq+idWBoVqaT8mjn\naT9Njtt5wH0TtOv6SeXiZNkJV4f9LfbxJsLJcUtC63TgkTj9CPC+xDa2NGoBs4H7gU8QTsK3Jcpf\ni9u5jhCU+oEdYr7b47H9B+AHwBDhBL0wbvf7CZ3V8Xd0XVw2I+o8HNOWxOnd4zrfjzqjwH8kyn9/\n1LmYEADqF0NrCRdpw8Cjje2AcAJendC5Ny67OE7X6+6OmLexrr8Tj8O2Ouygrt9PuMOYSzipjwKf\nmmRdJ+8G6nXU9JwVpx9I1NEi4LqJ2mxRL9Hby8zGAMxsE7BnkzyNDwjdS6jUTnUAZkn6uaSfSJrs\n06fNHliaN0ktCGPal0u6SdL/bpP3vYSDnaZMjXWURidVeSR9StI6wp3S36QoT7NjlkYHUhwzSX8E\nrDezW1totCrTVI5bFjSW6T7CFV2dxna9V4v16m2wMf0JYP+oO6OuBfwGeGGcfiGhy7O+DSW0Nkha\nCWwinPRmRM0Nie3uksjfZ2ZjZvYc4SS4V0w/AvgooW09TbibfipR1g3AzjHvvsDDZrY16jxKuAqf\nR7iqXi7ppsR29yBcrdfLX9cZBDab2da4jfWEu8rfBx6TdJmkW4DXx7z9hG6euk5yv3ZN1N09hPNL\nY13vHet13iTqejbhpVLrCO1yM9u7tDqt62ZtvGl7kbQ7IWhtTaS/mAko09tVs3wwbl8zOwT4U2Ap\nocF1szwAbzSz1wPHEq6qX990o9JfAc+Y2VenUqY2OqnKY2afNLN9gX8ndANNqjwpdKDhmEl6ScP+\n7Az8FXBOm+0X8oBlh0x2zHnjevV3WW9Ll7QL8A5C983THerXtczMXks4oc5ne2Bqlb8Zg4Qr9FpD\nnsb8ltBptkyE50HqbfMgQpdRs2PYTKdeRzsQTn7/bGYHE/r7D2+jk0QN30jaMZancR/TYIRgdhDh\nTvjFhIuDVu/TmqiuW7XxidKb1XVLigoMY5L6ASTVbwMb2UC4iqgzn3A72qlOPUpjZmsJEfr3JlHm\nZuWZ9OOmiTI9QPgRL2jMI+kUwo/hnR2UqbGO0uikKk+CrwH/a7LlSaHT7Ji9tiHL/oSTzy8lrY3b\n+oWkxpNYpsctIxrLtDfhqrhOq3a9gfEXNfV92QDsK2kmcBWhu+H6mL61rkVo90/E6ScIV9H1bVhS\nC8DMHiN048wlXPXWtz2f0B9ez/+IpP747MZcwjHfBThE0t2E4zyPELBmJfZ9PiF4bSRckc+VNCPq\nzCFc7W4AXhTL8wDhKrmf0I3yskT5n4o6a4E58SWGxDJvIHS5PGFmN8f0Bwh3HWOEdlTXSe7XY4m6\n2y8eh+SxWww8ROhimkxd7w78zswejncA9bvDTut6wnbRmG5mDwJ9iTpq+5vIKzA0RqhrCH1+EEaM\nXN1kneXAUZLmSJoLHAX8uFMdSX2SdorTewBvZLs506qszbgJeJmk/aLeyXH7zaLvhFqSZscrOyS9\nEDia0IeZvBo5hmAyHWdmrV7g0axMP+xUJ2V5XpZY5XhCF0MjbY9ZGp0Wx+w2EnVtZr82swEze6mZ\nvYTwI3itmTVeHDQr0/Jm9VAgjcftOEJfc51kux5me7u+BngPbHuyenPsulhO2K+vELpXBoDlMbg+\nBHxS4T0RHyYEDAjt5MNx+pPAg1HrJuAtsb72JpzQrgH+A5gp6RDCb21mLNc1hJP1MMHT2BDTP0AI\nUAcSRkxtIVyc/BBYIGk/4FRCd0pd58GocWIs99WxvO+UtJOkVxJOYt8jeEuvjjqnEU62dZ3HgRNj\nHe0c834d2FHSwbEdvJnQNr8FvCDu1zAhUNV1tgDDUWcW8E0S7Skei1lTqOvvAvtL2iuW6Q1xnU7r\n+hrg5FhHLyEEzJ/T/PxQb0s3RA1ofQ7eTg5G21cJ0WgLoS/tVEKk+yHBePoBod8M4HXAvyTWHSZE\n4zsIQ8o61gH+gDD0ayVhmGQznRMIVydPEfpl6+bY3sB3E+U5Jm5rDcHIabZvbbWAlxBuP1cSRhzU\nmuisIVxF3RI/X0hRpknppCzPVYllVwN7T/KYtdVpcsyGm9V1Qzu7m2g+tylTmYarriYYpY+Rol3H\n9T5PuLD5JYnBDQTTst6XPxqP9THAYYQT5e+ifv03shuhnf6OEJTeHNNfFdffEj9XJtrIrTHtt2wf\nfTOLMJz4SUKb/3FiGx+PZb2H7cNVX0K4eq9v9xMJne/F8j9FMHV3jPnrx/1pwoOC9fKvjDpPErqb\n6nWxjO3DbZcDO8b0v2b7MNOb2T7y6saY90nGG8v/GtOeJgztrO/XcNyvZ4mmckzvqK7jsm/F9C2x\n7ur7PNm6bjZcdds5K5H+krjfdxBGKO04UXv1B9wcx3GccZTJfHYcx3FKgAcGx3EcZxweGBzHcZxx\neGBwHMdxxuGBwXEcxxmHBwbHcRxnHB4YHMdxnHF4YHAcx3HG8f8B3rxJc/rZu/EAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108652cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "prices = pd.DataFrame({\"price\":train_df[\"SalePrice\"], \"log(price + 1)\":np.log1p(train_df[\"SalePrice\"])})\n",
    "prices.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可见，label本身并不平滑。为了我们分类器的学习更加准确，我们会首先把label给“平滑化”（正态化）\n",
    "\n",
    "这一步大部分同学会miss掉，导致自己的结果总是达不到一定标准。\n",
    "\n",
    "这里我们使用最有逼格的log1p, 也就是 log(x+1)，避免了复值的问题。\n",
    "\n",
    "记住哟，如果我们这里把数据都给平滑化了，那么最后算结果的时候，要记得把预测到的平滑数据给变回去。\n",
    "\n",
    "按照“怎么来的怎么去”原则，log1p()就需要expm1(); 同理，log()就需要exp(), ... etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train = np.log1p(train_df.pop('SalePrice'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后我们把剩下的部分合并起来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "all_df = pd.concat((train_df, test_df), axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此刻，我们可以看到all_df就是我们合在一起的DF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2919, 79)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "而*y_train*则是*SalePrice*那一列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Id\n",
       "1    12.247699\n",
       "2    12.109016\n",
       "3    12.317171\n",
       "4    11.849405\n",
       "5    12.429220\n",
       "Name: SalePrice, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: 变量转化\n",
    "\n",
    "类似『特征工程』。就是把不方便处理或者不unify的数据给统一了。\n",
    "\n",
    "#### 正确化变量属性\n",
    "\n",
    "首先，我们注意到，*MSSubClass* 的值其实应该是一个category，\n",
    "\n",
    "但是Pandas是不会懂这些事儿的。使用DF的时候，这类数字符号会被默认记成数字。\n",
    "\n",
    "这种东西就很有误导性，我们需要把它变回成*string*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_df['MSSubClass'].dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "all_df['MSSubClass'] = all_df['MSSubClass'].astype(str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "变成*str*以后，做个统计，就很清楚了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20     1079\n",
       "60      575\n",
       "50      287\n",
       "120     182\n",
       "30      139\n",
       "70      128\n",
       "160     128\n",
       "80      118\n",
       "90      109\n",
       "190      61\n",
       "85       48\n",
       "75       23\n",
       "45       18\n",
       "180      17\n",
       "40        6\n",
       "150       1\n",
       "Name: MSSubClass, dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_df['MSSubClass'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 把category的变量转变成numerical表达形式\n",
    "\n",
    "当我们用numerical来表达categorical的时候，要注意，数字本身有大小的含义，所以乱用数字会给之后的模型学习带来麻烦。于是我们可以用One-Hot的方法来表达category。\n",
    "\n",
    "pandas自带的get_dummies方法，可以帮你一键做到One-Hot。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass_120</th>\n",
       "      <th>MSSubClass_150</th>\n",
       "      <th>MSSubClass_160</th>\n",
       "      <th>MSSubClass_180</th>\n",
       "      <th>MSSubClass_190</th>\n",
       "      <th>MSSubClass_20</th>\n",
       "      <th>MSSubClass_30</th>\n",
       "      <th>MSSubClass_40</th>\n",
       "      <th>MSSubClass_45</th>\n",
       "      <th>MSSubClass_50</th>\n",
       "      <th>MSSubClass_60</th>\n",
       "      <th>MSSubClass_70</th>\n",
       "      <th>MSSubClass_75</th>\n",
       "      <th>MSSubClass_80</th>\n",
       "      <th>MSSubClass_85</th>\n",
       "      <th>MSSubClass_90</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    MSSubClass_120  MSSubClass_150  MSSubClass_160  MSSubClass_180  \\\n",
       "Id                                                                   \n",
       "1              0.0             0.0             0.0             0.0   \n",
       "2              0.0             0.0             0.0             0.0   \n",
       "3              0.0             0.0             0.0             0.0   \n",
       "4              0.0             0.0             0.0             0.0   \n",
       "5              0.0             0.0             0.0             0.0   \n",
       "\n",
       "    MSSubClass_190  MSSubClass_20  MSSubClass_30  MSSubClass_40  \\\n",
       "Id                                                                \n",
       "1              0.0            0.0            0.0            0.0   \n",
       "2              0.0            1.0            0.0            0.0   \n",
       "3              0.0            0.0            0.0            0.0   \n",
       "4              0.0            0.0            0.0            0.0   \n",
       "5              0.0            0.0            0.0            0.0   \n",
       "\n",
       "    MSSubClass_45  MSSubClass_50  MSSubClass_60  MSSubClass_70  MSSubClass_75  \\\n",
       "Id                                                                              \n",
       "1             0.0            0.0            1.0            0.0            0.0   \n",
       "2             0.0            0.0            0.0            0.0            0.0   \n",
       "3             0.0            0.0            1.0            0.0            0.0   \n",
       "4             0.0            0.0            0.0            1.0            0.0   \n",
       "5             0.0            0.0            1.0            0.0            0.0   \n",
       "\n",
       "    MSSubClass_80  MSSubClass_85  MSSubClass_90  \n",
       "Id                                               \n",
       "1             0.0            0.0            0.0  \n",
       "2             0.0            0.0            0.0  \n",
       "3             0.0            0.0            0.0  \n",
       "4             0.0            0.0            0.0  \n",
       "5             0.0            0.0            0.0  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(all_df['MSSubClass'], prefix='MSSubClass').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此刻*MSSubClass*被我们分成了12个column，每一个代表一个category。是就是1，不是就是0。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同理，我们把所有的category数据，都给One-Hot了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>BsmtUnfSF</th>\n",
       "      <th>...</th>\n",
       "      <th>SaleType_ConLw</th>\n",
       "      <th>SaleType_New</th>\n",
       "      <th>SaleType_Oth</th>\n",
       "      <th>SaleType_WD</th>\n",
       "      <th>SaleCondition_Abnorml</th>\n",
       "      <th>SaleCondition_AdjLand</th>\n",
       "      <th>SaleCondition_Alloca</th>\n",
       "      <th>SaleCondition_Family</th>\n",
       "      <th>SaleCondition_Normal</th>\n",
       "      <th>SaleCondition_Partial</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>196.0</td>\n",
       "      <td>706.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>284.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>162.0</td>\n",
       "      <td>486.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>434.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1915</td>\n",
       "      <td>1970</td>\n",
       "      <td>0.0</td>\n",
       "      <td>216.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>540.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2000</td>\n",
       "      <td>2000</td>\n",
       "      <td>350.0</td>\n",
       "      <td>655.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>490.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 303 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  YearRemodAdd  \\\n",
       "Id                                                                            \n",
       "1          65.0     8450            7            5       2003          2003   \n",
       "2          80.0     9600            6            8       1976          1976   \n",
       "3          68.0    11250            7            5       2001          2002   \n",
       "4          60.0     9550            7            5       1915          1970   \n",
       "5          84.0    14260            8            5       2000          2000   \n",
       "\n",
       "    MasVnrArea  BsmtFinSF1  BsmtFinSF2  BsmtUnfSF          ...            \\\n",
       "Id                                                         ...             \n",
       "1        196.0       706.0         0.0      150.0          ...             \n",
       "2          0.0       978.0         0.0      284.0          ...             \n",
       "3        162.0       486.0         0.0      434.0          ...             \n",
       "4          0.0       216.0         0.0      540.0          ...             \n",
       "5        350.0       655.0         0.0      490.0          ...             \n",
       "\n",
       "    SaleType_ConLw  SaleType_New  SaleType_Oth  SaleType_WD  \\\n",
       "Id                                                            \n",
       "1              0.0           0.0           0.0          1.0   \n",
       "2              0.0           0.0           0.0          1.0   \n",
       "3              0.0           0.0           0.0          1.0   \n",
       "4              0.0           0.0           0.0          1.0   \n",
       "5              0.0           0.0           0.0          1.0   \n",
       "\n",
       "    SaleCondition_Abnorml  SaleCondition_AdjLand  SaleCondition_Alloca  \\\n",
       "Id                                                                       \n",
       "1                     0.0                    0.0                   0.0   \n",
       "2                     0.0                    0.0                   0.0   \n",
       "3                     0.0                    0.0                   0.0   \n",
       "4                     1.0                    0.0                   0.0   \n",
       "5                     0.0                    0.0                   0.0   \n",
       "\n",
       "    SaleCondition_Family  SaleCondition_Normal  SaleCondition_Partial  \n",
       "Id                                                                     \n",
       "1                    0.0                   1.0                    0.0  \n",
       "2                    0.0                   1.0                    0.0  \n",
       "3                    0.0                   1.0                    0.0  \n",
       "4                    0.0                   0.0                    0.0  \n",
       "5                    0.0                   1.0                    0.0  \n",
       "\n",
       "[5 rows x 303 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_dummy_df = pd.get_dummies(all_df)\n",
    "all_dummy_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 处理好numerical变量\n",
    "\n",
    "就算是numerical的变量，也还会有一些小问题。\n",
    "\n",
    "比如，有一些数据是缺失的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LotFrontage     486\n",
       "GarageYrBlt     159\n",
       "MasVnrArea       23\n",
       "BsmtHalfBath      2\n",
       "BsmtFullBath      2\n",
       "BsmtFinSF2        1\n",
       "GarageCars        1\n",
       "TotalBsmtSF       1\n",
       "BsmtUnfSF         1\n",
       "GarageArea        1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_dummy_df.isnull().sum().sort_values(ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，缺失最多的column是LotFrontage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "处理这些缺失的信息，得靠好好审题。一般来说，数据集的描述里会写的很清楚，这些缺失都代表着什么。当然，如果实在没有的话，也只能靠自己的『想当然』。。\n",
    "\n",
    "在这里，我们用平均值来填满这些空缺。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LotFrontage        69.305795\n",
       "LotArea         10168.114080\n",
       "OverallQual         6.089072\n",
       "OverallCond         5.564577\n",
       "YearBuilt        1971.312778\n",
       "YearRemodAdd     1984.264474\n",
       "MasVnrArea        102.201312\n",
       "BsmtFinSF1        441.423235\n",
       "BsmtFinSF2         49.582248\n",
       "BsmtUnfSF         560.772104\n",
       "dtype: float64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_cols = all_dummy_df.mean()\n",
    "mean_cols.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "all_dummy_df = all_dummy_df.fillna(mean_cols)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看看是不是没有空缺了？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_dummy_df.isnull().sum().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 标准化numerical数据\n",
    "\n",
    "这一步并不是必要，但是得看你想要用的分类器是什么。一般来说，regression的分类器都比较傲娇，最好是把源数据给放在一个标准分布内。不要让数据间的差距太大。\n",
    "\n",
    "这里，我们当然不需要把One-Hot的那些0/1数据给标准化。我们的目标应该是那些本来就是numerical的数据：\n",
    "\n",
    "先来看看 哪些是numerical的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['LotFrontage', 'LotArea', 'OverallQual', 'OverallCond', 'YearBuilt',\n",
       "       'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF',\n",
       "       'TotalBsmtSF', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea',\n",
       "       'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr',\n",
       "       'KitchenAbvGr', 'TotRmsAbvGrd', 'Fireplaces', 'GarageYrBlt',\n",
       "       'GarageCars', 'GarageArea', 'WoodDeckSF', 'OpenPorchSF',\n",
       "       'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 'MiscVal',\n",
       "       'MoSold', 'YrSold'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_cols = all_df.columns[all_df.dtypes != 'object']\n",
    "numeric_cols"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算标准分布：(X-X')/s\n",
    "\n",
    "让我们的数据点更平滑，更便于计算。\n",
    "\n",
    "注意：我们这里也是可以继续使用Log的，我只是给大家展示一下多种“使数据平滑”的办法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "numeric_col_means = all_dummy_df.loc[:, numeric_cols].mean()\n",
    "numeric_col_std = all_dummy_df.loc[:, numeric_cols].std()\n",
    "all_dummy_df.loc[:, numeric_cols] = (all_dummy_df.loc[:, numeric_cols] - numeric_col_means) / numeric_col_std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: 建立模型\n",
    "\n",
    "#### 把数据集分回 训练/测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dummy_train_df = all_dummy_df.loc[train_df.index]\n",
    "dummy_test_df = all_dummy_df.loc[test_df.index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1460, 303), (1459, 303))"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy_train_df.shape, dummy_test_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ridge Regression\n",
    "\n",
    "用Ridge Regression模型来跑一遍看看。（对于多因子的数据集，这种模型可以方便的把所有的var都无脑的放进去）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "这一步不是很必要，只是把DF转化成Numpy Array，这跟Sklearn更加配"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = dummy_train_df.values\n",
    "X_test = dummy_test_df.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "用Sklearn自带的cross validation方法来测试模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "alphas = np.logspace(-3, 2, 50)\n",
    "test_scores = []\n",
    "for alpha in alphas:\n",
    "    clf = Ridge(alpha)\n",
    "    test_score = np.sqrt(-cross_val_score(clf, X_train, y_train, cv=10, scoring='neg_mean_squared_error'))\n",
    "    test_scores.append(np.mean(test_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "存下所有的CV值，看看哪个alpha值更好（也就是『调参数』）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEKCAYAAAD5MJl4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHAVJREFUeJzt3XmQHOWd5vHvIyQOCd0gdCFgYDiGAGzQilND2zCDsAkO\nG48lW4Nw2Ovx4mvWx4CJtRHL2DNEGFsw2rGxFxjMpTFGmMMODAz0OjAGxH1JCINDtwQ6kITORvrt\nH5kllSqrukrq7qrKrucTkdFVmVmZb2Vkv0++bx6liMDMzKxYn0YXwMzMmo/DwczMMhwOZmaW4XAw\nM7MMh4OZmWU4HMzMLMPhYE1D0i2S/nd3z2tmu8/hYHUnqV3Sakn9Gl2W7iRpoKQZkhZIWidpvqQf\nSRou6SFJ08t85gJJyyRl/hfT7bQpXVZhuK8uX8ZansPB6krSIcAZwHbg/AYXp9ukQfcYcAzwtxEx\nCDgNWAX8N+A/gEvKfHQqcFtEbC8zLYDLImJQ0XBBhfXvVcu4Kt9ht+a33s3hYPV2CfBHksry0koz\nSTpT0iJJ35H0rqS3JX2mZLZhkh5Mj6j/KOmwos/PkLRQ0lpJcySdUWE9J6dH7ioad5Gkl9LXE9LP\nr03n+2GFIk8DxgIXRsQbABGxMiK+HxEPAb8GhhaXQ9IQ4DzgF5W2A6CyI3dun3+StAy4udy4dN7/\nLulNSSsl/VrSqKLlbJd0maT5wPxOymEtxuFg9XYJcDtwJ3COpAM7mXckMAwYTRIkP5P0l0XTJwNX\nAUOAt4DvF017BjgeGJqu625Je5euICKeBt4HPlo0ekpaRoAZwIyIGAwcDvyyQlnPAh6KiE3lJkbE\nZuBudm09fBqYGxGvVlhmNSNJvvs44Ivlxkn6KPAD4GJgFLAQmFWynAtIWjd/tYflsF7I4WB1kx41\njwN+GRHPA38CSlsDxQL4bkR0RMTvgd8Af1c0fXZEPJd2ydwBfGjHByPujIj3ImJ7RPwY2Ac4qsJ6\nZhXKIWkg8DHgrnTaVuAIScMjYmNEPFNhGcOBZZ18F4Bbgb+TtE/6/u/TcZ35t/T8zJr079VF07YB\nV6XbZ0uFcZ8BboqIlyKiA/gOcKqkcUXL+UFErC1ahpnDwerqEuDhiFiTvr+LpDumkjXpEXfBApJW\nRMHyotcbgf0LbyR9U9LraaW6BhgEHFBhPXcCF6XnDT4BPBcRi9NpnycJlXmSnpb08QrLWEVyZF5R\nRPwBeAe4IO0CG5+uuzNfjYhhETE0/XtV0bR30wqfTsaNJtluhTJsSMs6pmiexZiV6NvoAlhrkLQv\nyVF/n7Q/HGBvYIik4yLilTIfGyppv6KumnFAuflK1zUR+CfgIxHxejpuNRX67yNirqQFJC2GKRRV\n2BHxFjtbFZ8EfiVpWJnuo0eBa0rKW85tJIF4NElQvlvt+3Si3COVS8ctBQ4pvJE0gKSVs7iTz5i5\n5WB1cxHwAcnVPCekwzHAE5S/igeSyvxqSf3SCv/jVO7zL7Y/0AGskrS3pO8BA6t85k7ga8BEknMD\nSQGkz0oqtDjWklSk28p8/jZgEXCPpKOUGJ6eUJ9UNN8vgLOBL1C9S6k73Al8TtLxaXfWD4CnImJR\nHdZtOeZwsHq5BLg5IpZExDuFAZgJfLbcdf4kffhrSI5+bwP+ISLeTKd1drT7O+Ahkqtv/kzS5VSt\nMpwFnAn8V0SsLho/CXhN0jrgx8CnI2Jr6YfTcWcD84BHSILkKZKj9KeL5lsAPAn0B+6vUiaAmUX3\nOKyXNKeGzxSX6zHgu8BsYAlwGMmJ/B2z7M7yrHWolh/7SY98ZpCEyU0RcW3J9Inp9ONJ/nlmF03b\nBrxEchS4ICIuTMffQvLPWDgauzQiXu6OL2X5J+lMkuv/x1Wd2cy6XdVzDukR3UySS/WWAnMk3RcR\n84pmW0DSj/qtMovYEBEnVlj8NyPi3t0ss5mZ9bBaTkhPAN5Mm8NImkVyXfSOcIiIhem0cs2QsicB\nU+7WMjNrQrVUzmPYtb92MbteBlfNPpKekfSkpNJb//9Z0ouSrlMve86OdU1E/D93KZk1Ti3hUO7I\nf3dOYo2LiAnAZ4EZRY84uCIijiG5M3M4cPluLNPMzHpQLd1Ki0muLy8YS3LuoSYRsTz9+2dJ7cCH\ngT9HxIp0fEd6cvqb5T5foavKzMyqiIjOuvU7VUvLYQ7J4wMOSZ9NM5nOL8ErfoDZkMLzbNJrxU8D\nCjcljUz/CrgQqPh8mYjwEMFVV13V8DI0y+Bt4W3hbdH50FVVWw4RsU3SV4CH2Xkp69z0GS9zIuJB\nSeOBe0ke+HWepOkRcRzJTU43ppez9gH+JXZe5XRHGhgCXgS+1OVvY2Zm3aKmx2dE8sjho0rGXVX0\n+lng4DKf+yPJvQ/llnnWbpXUzMzqxpeS5khbW1uji9A0vC128rbYydui+9R0h3QjSYpmL6OZWbOR\nRPTwCWkzM2sxDgczM8twOJiZWYbDwczMMhwOZmaW4XAwM7MMh4OZmWU4HMzMLMPhYGZmGQ4HMzPL\ncDiYmVmGw8HMzDIcDmZmluFwMDOzDIeDmZllOBzMzCzD4WBmZhkOBzMzy3A4mJlZhsPBzMwyHA5m\nZpbhcDAzswyHg5mZZTgczMwsw+FgZmYZDgczM8twOJiZWYbDwczMMhwOZmaW4XAwM7MMh4OZmWU4\nHMzMLMPhYGZmGQ4HMzPLcDiYmVmGw8HMzDIcDmZmluFwMDOzDIeDmZllOBzMzCzD4WBmZhkOBzMz\ny3A4mJlZhsPBzMwyHA5mZpZRUzhImiRpnqT5ki4vM32ipOckdUj6RMm0bZKel/SCpF8XjT9U0lOS\n3pB0l6S+Xf86ZmbWHaqGg6Q+wEzgHOBYYIqko0tmWwBMA+4os4gNEXFiRHw4Ii4sGn8tcF1EHAW8\nB3x+T76AmZl1v1paDhOANyNiQUR0ALOAC4pniIiFEfEqEGU+rwrL/ShwT/r6VuCi2opsZmY9rZZw\nGAMsKnq/OB1Xq30kPSPpSUkXAEgaDqyJiO1Fyxy9G8s0M7MeVEs/f7kj/3IthErGRcRySYcBj0l6\nGVhfZrkVlzl9+vQdr9va2mhra9uN1ZuZ9X7t7e20t7d32/IU0Xk9L+kUYHpETErfXwFERFxbZt5b\ngAciYnaFZe2YLukdYGREbE/XcVVEnFvmM1GtjGZmtitJRESlbv2qaulWmgMcIekQSXsDk4H7OytT\nUeGGpJ9B0gHA6cDr6eTHgU+lr6cB9+1m2c3MrIdUbTlAcikrcD1JmNwUEf8q6WpgTkQ8KGk8cC8w\nBNgMLI+I4ySdCtwIbEs/++OI+I90mYeRnNweCrwATE1PeJeu2y0HM7Pd1NWWQ03h0EgOBzOz3VeP\nbiUzM2sxDgczM8twOJiZWYbDwczMMhwOZmaW4XAwM7MMh4OZmWU4HMzMLMPhYGZmGQ4HMzPLcDiY\nmVmGw8HMzDIcDmZmluFwMDOzDIeDmZllOBzMzCzD4WBmZhkOBzMzy3A4mJlZhsPBzMwyHA5mZpbh\ncDAzswyHg5mZZTgczMwsw+FgZmYZDgczM8twOJiZWYbDwczMMhwOZmaW4XAwM7MMh4OZmWU4HMzM\nLMPhYGZmGbkIh4hGl8DMrLXkIhy2bWt0CczMWksuwqGjo9ElMDNrLbkIhw8+aHQJzMxaSy7CYevW\nRpfAzKy15CIc3K1kZlZfDgczM8vIRTi4W8nMrL4cDmZmlpGLcHC3kplZfeUiHNxyMDOrL4eDmZll\n5CIc3K1kZlZfuQiHLVsaXQIzs9ZSUzhImiRpnqT5ki4vM32ipOckdUj6RJnpAyUtlnRD0bjH02W+\nIOl5SQdUWr+7lczM6qtvtRkk9QFmAmcBS4E5ku6LiHlFsy0ApgHfqrCYa4D2MuOnRMQL1crgloOZ\nWX3V0nKYALwZEQsiogOYBVxQPENELIyIV4HMLy9IOgkYATy8h+t3OJiZ1VktlfMYYFHR+8XpuKok\nCfgh8G1AZWa5Oe1S+l+dLcfhYGZWX1W7lShfqdf622yXAb+JiCVJTuyyrM9ExDJJA4DZkqZGxO3l\nFnLPPdNZsCB53dbWRltbW42rNzNrDe3t7bS3t3fb8hRVfoNT0inA9IiYlL6/AoiIuLbMvLcAD0TE\n7PT97cAZwHZgINAP+PeIuLLkc9OAkyLia2WWGTNmBF//+p58PTOz1iSJiCh3cF+TWloOc4AjJB0C\nLAMmA1M6K1PhRURMLSpoIQCulLQXMCQiVknqB5wHPFJpge5WMjOrr6rnHCJiG/AVkhPKrwGzImKu\npKslnQcgabykRcDFwE8lvVJlsfsAv5P0IvA8yXmMn1ea2eFgZlZfVbuVGk1SXHll8P3vN7okZmb5\n0dVupVzcIb15c6NLYGbWWnIRDps2NboEZmatJRfh4JaDmVl95SIc3HIwM6uvXISDWw5mZvWVi3Bw\ny8HMrL4cDmZmlpGLcHC3kplZfeUiHNxyMDOrr1yEw8aNjS6BmVlryUU4bNjQ6BKYmbWWXISDWw5m\nZvWVi3DYsAGa/PmAZma9Si7CoU8f2Lq10aUwM2sduQiH/v193sHMrJ5yEQ4DBvi8g5lZPeUmHNxy\nMDOrn1yEg7uVzMzqKxfhsP/+Dgczs3rKRTgMHAjr1ze6FGZmrSM34bBuXaNLYWbWOnIRDoMGueVg\nZlZPuQgHtxzMzOorF+HgloOZWX3lIhzccjAzq69chINbDmZm9ZWLcPClrGZm9ZWLcBg0yN1KZmb1\nlItwGDwY3nuv0aUwM2sduQiHoUNhzZpGl8LMrHXkIhyGDXM4mJnVUy7CodBy8E+FmpnVRy7CYd99\nk58K9Q/+mJnVRy7CAXzewcysnnITDj7vYGZWP7kJB7cczMzqx+FgZmYZuQmHYcNg1apGl8LMrDXk\nJhxGjIB33210KczMWkOuwuGddxpdCjOz1pCbcDjoIFixotGlMDNrDbkJB7cczMzqx+FgZmYZuQkH\ndyuZmdWPosmfZicpIoKODujfH7ZsSZ6zZGZmlUkiIrSnn89NNduvX/KLcL4Rzsys5+UmHCA57+Cu\nJTOznldTOEiaJGmepPmSLi8zfaKk5yR1SPpEmekDJS2WdEPRuBMlvZwuc0Yt5Rg7FhYvrmVOMzPr\niqrhIKkPMBM4BzgWmCLp6JLZFgDTgDsqLOYaoL1k3E+AL0TEkcCRks6pVpaDD4aFC6vNZWZmXVVL\ny2EC8GZELIiIDmAWcEHxDBGxMCJeBTJntyWdBIwAHi4aNxIYGBHPpKN+AVxYrSDjxsGiRTWU2MzM\nuqSWcBgDFFfJi9NxVUkS8EPg20DxWfMx6XJ2a5luOZiZ1UffGuYpdylUrde/Xgb8JiKWJDmxZ8uc\nPn06AG+9Ba+/3ga01bh6M7PW0N7eTnt7e7ctr+p9DpJOAaZHxKT0/RVARMS1Zea9BXggIman728H\nzgC2AwOBfsC/AzcAj0fEMel8k4EzI+J/lFlmFMo4bx6cfz7Mn7+H39bMrEV09T6HWloOc4AjJB0C\nLAMmA1M6K1PhRURMLSroNOCkiLgyfb9O0oR0+ZeQBEanDj44OecQAdrjr2xmZtVUPecQEduAr5Cc\nUH4NmBURcyVdLek8AEnjJS0CLgZ+KumVGtZ9GXATMJ/khPdD1T4wYEByl/TKlTUs3czM9lhuHp9R\n8OEPw89/DuPHN7BQZmZNrmUen1Fw5JE+52Bm1tNyFw5HH52cmDYzs56Tu3A46ih4441Gl8LMrHfL\nXTi45WBm1vNyd0L6/feTp7OuXw977dXAgpmZNbGWOyG9//4wfLgfo2Fm1pNyFw6QdC35vIOZWc/J\nbTjMndvoUpiZ9V65DIcTToAXXmh0KczMeq9chsOECfD0040uhZlZ75W7q5UAtm2DIUOSk9JDhzao\nYGZmTazlrlaC5BLWE0+EZ59tdEnMzHqnXIYDwMknu2vJzKyn5DYcJkyAZ56pPp+Zme2+3IZDoeXQ\n5KdMzMxyKbfhMHZscu7h7bcbXRIzs94nt+EgwaRJ8NvfNrokZma9T27DAeC88+DBBxtdCjOz3ieX\n9zkUrF8Po0fD0qUwcGCdC2Zm1sRa8j6HgoED4dRT4ZFHGl0SM7PeJdfhAO5aMjPrCbnuVgJ46y04\n7TRYtgz65D7qzMy6R1e7lfp2Z2Ea4fDD4cAD4Ykn4K//utGlMTOrn82bYfny5OC4+O/y5V1fdu5b\nDgDXXw9PPgn/+Z91KpSZWQ+JgFWrspV9aQAsWwabNsHIkckwatSuf7/0pa61HHpFOKxdC4ceCq++\nCmPG1KdcZma7Y/NmWLGifCVfHAArViQ/h1yo5MtV/IW/Q4cm93yV09VupV4RDgBf/jIMGwbXXFOH\nQpmZkRzlr15d21H+hg2VK/vS1/vs0/WyORxSc+fCRz4CCxZ0z4Y1s9a1ZUvtR/n9+3d+lF94PXRo\nfS+acTgU+Zu/gWnTYOrUHi6UmeVOBKxZU9tR/vvvw0EHVT66Lx63776N/mblORyK3H8/fO978Nxz\nyUP5zKz327q1tqP85cthv/0678MvvB42LP+XxjscikRAWxtcfDF89as9Wy4z61kdHUmlv3Rp58O6\ndTBiRPWTtyNHJuHQKhwOJebOhYkT4aWXfOWSWTPatg3efbd6pb9qVVLpjx5deRg1Cg44IP9H+T3B\n4VDGd78L8+bB3Xf3UKHMLKNwfX61Sv+dd2DIkKRyHzOmcsU/YoS7h7vC4VDGpk1w3HFwww3wsY/1\nUMHMWkREci9RtUp/2TIYMKDzI/3Ro5Punb33bvS36v0cDhU88gh88Yvw4osweHAPFMysF3j//eqV\n/tKl0Ldv9Up/1KjW6tNvdg6HTvzjP8Lzz8NDDyXXIpu1ii1bkiP5JUuyFX3xuI6Ozrt2CpW+fy8l\nfxwOndi+PbnvYfVquPdeN2Ut/wr9+oUKfsmS8sPatUn3TaHirxQAgwdXfvyC5ZvDoYqODvjkJ5O+\n0Ntv9wkua16bN+9a4Zer/JcuTbpuxozZdSgEQGEYMcJX8LQ6h0MNNm+Gc8+Fo46Cn/zER0pWXxGw\ncmW2ki+t+NetS7pwSiv+0srfXaRWC4dDjdatSx6vMXYs3Hhjcm20WVdt2pSt6EvfL1uWVOjlKv3i\niv/AA320b93H4bAbNm9O7oG44w742c+Snxg1K6dwo1ZxZV/6d8mS5GqfSkf7hWHUKB/tW/05HPbA\n73+fnKg++2z40Y98JUYriUhakZ1V+kuXJo9tKL1Rq1zf/vDhPtq35uRw2EPr1sE3vgGPPQYzZ8Kk\nSf4nz7stW7KXa5b7K2W7dEr/dtcz9c0axeHQRQ88kHQ1bdqU/GDQtGm+aa7ZFD+Lp7NKf9268pdv\nlv4dNKjR38is5zkcukEE/OEPSQvi4Ydh8uQkKI49tkdX29IiYP36nY9ULn20cvGwcmXyQymFa/Mr\nVfo+oWu2k8Ohmy1blpysvvHG5NLXCy+EM86AE05IHiFgndu6NXmwWrUKf/nyZP5Ro3Z9pHLxUBh/\n4IHQr19jv5dZ3jgcesjWrcmPBz36KDzxBCxcCCefDKefnoTFKackPwLe23V0JHeYr1q1c1i5cuff\n0hBYu3bXZ+uX+33cwtAK28+sUeoSDpImATOAPsBNEXFtyfSJ6fTjgU9HxOx0/Dhgdvq5fsDMiLgx\nnfY4MArYBATwtxGxssy6GxIOpVavhiefTILiiSfghRfgmGNg/Pjk3onSyxeb7bEEEckPnBdX8qWV\nfblh48akS2f48OxwwAHZSt/P1jdrDj0eDpL6APOBs4ClwBxgckTMK5pnHDAI+BZwf1E49E3X0SGp\nP/AacGpELE/D4RsR8UKV9TdFOJTavBmefTZ56mu5Z9ts374zKArhMXhw0jXVr18y1PK6T59kXRs3\nwrPPtnPooW1s3MiOYdMmdnlfbtiwIQm3vn3LV/KlFX7x+0GDmrOyb29vp62trdHFaAreFjt5W+zU\n1XCopRd9AvBmRCxIVzgLuADYEQ4RsTCdtkstHhEfFL3dDygtaBNWO7XZd9+ke+mMM8pPX7++/OMR\nPvgg6aopDMXvy03bti15lk7//vCnP7Vzyilt9O/PjmHIEHZ5X5i3dBg2rHl/CH1PuBLYydtiJ2+L\n7lNLOIwBFhW9X0wSGDWRNBb4DXA48O2IWF40+WZJ24DZEfHPtS4zDwYOhKOPTobuMn16MpiZ9bRa\njtzLNUtq7ueJiMURcQJwBHCppAPTSZ9Jx08EJkqaWusyzcysZ9VyzuEUYHpETErfXwFE6UnpdNot\nwAOFcw5lpt8MPFg6XdI04KSI+FqZzzTfCQczsxzo6XMOc4AjJB0CLAMmA1M6mX9HYSSNAVZFxGZJ\nQ4HTgesk7QUMiYhVkvoB5wGPlFtYV76cmZntmd25lPV6dl7K+q+SrgbmRMSDksYD9wJDgM3A8og4\nTtLZwHXAdpLQ+LeIuCm9cun3JOG0F/AoyZVLbiWYmTWBpr8JzszM6q9pLyWVNEnSPEnzJV3e6PLU\nk6Sxkh6T9LqkVyR9LR0/VNLDkt6Q9DtJLfOIQEl9JD0v6f70/aGSnkq3xV3pPTW9nqTBku6WNFfS\na5JObtX9QtL/lPSqpJcl3SFp71bZLyTdJGmFpJeLxlXcDyTdIOlNSS9K+lAt62jKcEhvvJsJnAMc\nC0yR1I0XhTa9D0i62f4KOBX4cvr9rwAejYijgMeA7zSwjPX2deD1ovfXAtel2+I94PMNKVX9XQ/8\nNiKOAU4gud+o5fYLSaOBrwInRsTxJF3UU2id/eIWkvqxWNn9QNK5wOER8ZfAPwA/rWUFTRkOFN14\nFxEdQOHGu5YQEcsj4sX09fvAXGAsyTa4NZ3tVuDCxpSwvtJ7ZT4G/N+i0R8F7klf3wpcVO9y1Zuk\ngcDEiLgFkptMI2ItLbpfkJyvHJC2DvYjeYLDR2iB/SIingDWlIwu3Q8uKBr/i/RzTwODJR1UbR3N\nGg7lbrwb06CyNJSkQ4EPAU8BB0XECkgCBDiw8id7lR8D3ya9v0bScGBNRGxPpy8GRjeobPX0F8BK\nSbekXWw/Sy/uaLn9IiKWklzsshBYAqwFngfea8H9omBEyX4wIh1fWp8uoYb6tFnDoUs33vUWkvYH\nfgV8PW1BtOI2+DiwIm1JFfYLkd1HWmHb9AVOBP5PRJwIbCDpSmiF774LSUNIjogPIQmAAcC5ZWZt\nuW1Txh7Vp80aDouBcUXvx5I0GVtG2lT+FXBbRNyXjl5RaA5KGgm806jy1dHpwPmS3gbuIulOmkHS\nNC7sv62yfywGFkXEs+n7e0jCohX3i7OBtyNidURsI7mU/jRgSAvuFwWV9oPFwMFF89W0XZo1HHbc\neCdpb5Ib7+5vcJnq7Wbg9Yi4vmjc/cCl6etpwH2lH+ptIuLKiBgXEX9Bsh88FhFTgceBT6Wztcq2\nWAEsknRkOuoskicdt9x+QdKddIqkfSWJnduilfaL0hZ08X5wKTu/+/3AJbDjiRfvFbqfOl14s97n\nUO7GuwYXqW4knU5yk+ArJM2/AK4EngF+SXIUsBD4VES816hy1pukM4FvRsT5kg4juVBhKPACMDW9\neKFXk3QCyYn5fsDbwOdITsy23H4h6SqSA4YOkn3gCyRHxb1+v5B0J9AGDAdWAFcBvwbupsx+IGkm\nMImkK/JzEfF81XU0aziYmVnjNGu3kpmZNZDDwczMMhwOZmaW4XAwM7MMh4OZmWU4HMzMLMPhYGZm\nGQ4HMzPL+P8xqzA0mSbmTQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108652278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.plot(alphas, test_scores)\n",
    "plt.title(\"Alpha vs CV Error\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "可见，大概alpha=10~20的时候，可以把score达到0.135左右。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "max_features = [.1, .3, .5, .7, .9, .99]\n",
    "test_scores = []\n",
    "for max_feat in max_features:\n",
    "    clf = RandomForestRegressor(n_estimators=200, max_features=max_feat)\n",
    "    test_score = np.sqrt(-cross_val_score(clf, X_train, y_train, cv=5, scoring='neg_mean_squared_error'))\n",
    "    test_scores.append(np.mean(test_score))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcXfP9x/HXO8T2E6JFQyypPdQSIqhtbBUq1RJLWm1S\nlFqqKraq/hLlp5Sqragitmoa+1Zbw1RRJAQRS2LJMrW1iKURsnx+f3zPjGtyJ3MzM3fOvXPfz8cj\nj8w959xzPvdOcj7nuysiMDMzA+iWdwBmZlY5nBTMzKyJk4KZmTVxUjAzsyZOCmZm1sRJwczMmjgp\nmJlZEycF+wJJUyXNlvSlZtufkTRf0hodfL01s/N+mP35SNKEDjjvCEnXdkSMlUJSd0kjJU3OvqfX\nJF2RfYeXSbqmyHs2yX6fPYvsGyXp047+7q26OSlYcwG8Dgxp3CDpa8BS2b5yXXP5iFguInpERL8y\nXadkkhbLO4Yibgb2Ag4Elgc2BZ4CdgauBr4jaelm7/k+cGdEzGzhnGdn3/tCv/ti38eifkcV+p1a\nM04KVsx1wNCC10OBLzyFStpT0tOSPpA0TdKIgn37S3pV0rLZ6z0kvSnpywu5populA6W9IKkdyXd\nU1hSkXS+pOlZDOMkbZdt3x04BTig8OlX0uuSdi54/whJ12U/N5ZYDpY0DRibbd9a0qOS3pc0QdKO\nBe8fln3OD7O/mxJpwTGrSJpV+KQuqZ+kf0taTNLakuolzZT0jqQ/t/A97ArsAnwrIp6OiPkR8VFE\nXBoRoyLiceBfwL4F7+kGfJeUMBZJse9jId/RtyQ9L+k9SQ9K2qDgPK9LOlHSs8DHWUxWwfwLsmIe\nB3pIWj/7T7w/cD1fvHF/DHw/IpYHvgn8WNK3ACJiDPAYcGFWDXUFcHBEvLuQay6QFCR9GzgZ+Daw\nEvAPoPCm+SSwCbACcANwo6QlIuI+4EzgLyWUPJqXfnYANgB2l7QqcBfwq4hYATgeuFnSlyUtA1wA\n7B4RywFfB55Z4OQRb2bfxb4Fm4cAYyJiHnA6cF9E9ARWAy5qIc5dgCcj4o2FfJbmyXw3YHHg3oW8\npzVN30exbZLWJX33x5B+R/cAd0pavOD4A4E9gJ4RMb8dsVgncFKwljTeYHYDXgK+cDOKiIcjYlL2\n8/PAaGDHgkOOJt3I6oHbI+KehVxLwL+zp/H3JB2XbT8M+HVETM5uJmcBm0laPbvuDRExM3tq/h2w\nJLB+Oz5zACMi4pOI+BQ4CLg7SzJExFhgPLBndvw8YGNJS0XE2xHxYgvn/TPpib3RgaQbKcAcYE1J\nvSPis4h4rIVzfBl4s5X4rwN2yJIZpKqjG7Lk05ITsu+88bsfVbCv+fdRbNsBwF0R8WB2nXOBpUlJ\nstEFEfFGwTmsgjkpWEuuJ93IhgELNNhK2iqrKnhH0kzgcGDFxv0R8QFwI7ARcF4r1wrgyxGxQkR8\nKSIaj18TuCC7Wb0HvJsd2zuLYXhWtfS+pPeB5QpjaKOGgp/XBPZvvH52jW2BVSJiFumGeATwpqQ7\nJbWUkG4CtpbUK6t+mh8Rj2b7TiD9P3xS0kRJP2zhHO8Cqyws8IiYQSpNHSTpf0glrKtb+bznZN95\n43ff/PoNRd5TuG1VYFpBDAHMIPsdLeQcVqGcFKyoiJhOanDeA7ilyCF/Am4DemdVH3+goApI0mbA\nwaSn5JaqRAoVa1OYDhye3awab1zLRsTjWfvBicDgbPsKwIcF5ynWKP5fYJmC172KHFP4vhnAtc2u\n3yMifgMQEQ9ExDey87wM/LHYB8sS5P2kJDKEgiqwiHgnIg6LiN7Aj4FLJK1V5DR/AwYUlAJacg2p\nhLcv8FpEPNvK8a0p9j0WbnuDlDwLrc4XE4GnYq4iTgq2MAcDO0fEJ0X2LQu8HxFzJA2goHpE0lKk\nqoyTs3OsKumIhVynaCMzKdGcImnD7LzLSxqc7etBqnp5V9ISkv4329bobaCPpMJzPwMcKGlxSf2B\nwXxR8ziuBwZJ+oakbpKWkrSjpFUlrSxpUNa2MIfUxjJ3IZ/xz8APgH34vOoISYMlNT5VzwTmk6ql\nviCrunoAuFXS5lkj9bKSDpc0rODQm0k35dNo1jmgDYr9XppvGwN8U9JO2fd6PDAb+Gc7r205cVKw\n5pqe6iLi9Yh4utg+4EjgdEkfAKcCfynYdyYwPSIuj4jPSHXbp0tau7VrfmFjxG2kdoTRWRXVc8DA\nbPd9pAbUyaQSzSzSk32jG0k3sHcljc+2/RJYB3gPGEEq7bQYR0Q0AHuTejL9m1RNcjzp/003YDip\nx89/SI2vR7bw+QDuANYF3oqIiQXbtwSekPQhqeR1TERMK3YCUhL7K+m7nglMBLYglSIaY55FSgy9\ni3y+Yk7UF8cpvFOwr7VSAhExmdT2cjHpO/omMCgi5hY73iqfSllkR9JA4HzSf4QrI+LsZvu3z/Zv\nAhwQEbc0298DeBG4JSKOybbdQyp2L06qBz0qvOKPmVmuWi0pZF0SLyZ1SdsIGFLYDzkzjVSP2dKT\nyemkXiiF9ouIfhGxMbAysN8ixG1mZmVQSvXRAGBKREyLiDmkrod7Fx4QEdOzbokLPOlL2oJ007+/\n2Xs+zvZ3B5Yo9l4zM+tcpSSF3nyxrraBL3Y3a1HWyHcuqdtdscFJ9wJvkXqN3FTKOc3MrHxKSQrF\neiCU+lR/JGnwz7+KnSsiBpL6Xi9Jmr/FzMxytHjrh9AAFM6MuRrNRrcuxDbAdpKOJHUX7C7po4g4\npfGAiPhM0p2kKqmxzU8gydVKZmZtEBEtdfduUSklhXHAOtlkWEuQhujfsZDjm4KIiIMiok9ErEXq\nyndtRJwi6X8k9QLI5kjZkzSVQlERUVF/RowYkXsMjqlrxeWYHFNH/2mrVpNCpPlMjiY1FE8CRkfE\ni5JOk7RXdmPvL2kGqR/1ZZImtnxGAP4HuEPSM8AE0kCjy9r8KczMrEOUUn1ERNxLs4nGImJEwc/j\nSaMoF3aOa8hGWEbEO6ReTWZmVkE8orkN6urq8g5hAY6pdJUYl2MqjWMqv5JGNOdJUlR6jGZmlUYS\nUaaGZjMzqxFOCmZm1sRJwczMmjgpmJlZEycFMzNr4qRgZmZNnBTMzKrcrFkddy4nBTOzKhYBO+0E\nDz/cMedzUjAzq2JjxsC8ebDddh1zPo9oNjOrUp9+ChtuCFdckUoLhTyi2cysxlx6KfTtu2BCaA+X\nFMzMqtDMmbDeevDQQ7DRRgvud0nBzKyG/PrXsPfexRNCe7ikYGZWZaZNg803h4kTYdVVix/jkoKZ\nWY049VQ46qiWE0J7lLTympmZVYYJE+Bvf4PJk8tzfpcUzMyqRASccAKMGAE9epTnGk4KZmZV4r77\noKEBDjmkfNdwUjAzqwLz5qVSwtlnQ/fu5buOk4KZWRW45hro2RO+9a3yXsddUs3MKtysWWmg2s03\nw1ZblfYed0k1M+uifvc7+PrXS08I7eGSgplZBXvnnTTp3RNPwNprl/6+tpYUnBTMzCrY0UfD4ovD\n+ecv2vvamhSqYvDaJ5/A0kvnHYWZWeeaPBn+8hd48cXOu2ZVtClcf33eEZiZdb6f/xyOPx5WXLHz\nrllSUpA0UNJLkiZLOqnI/u0lPSVpjqR9iuzvIalB0oXZ66Ul3SXpRUkTJZ25sOufdx7Mn1/qRzIz\nq36PPgrjxsExx3TudVtNCpK6ARcDuwMbAUMkbdDssGnAUOBPLZzmdKC+2bZzIqIv0A/YTtLuLcWw\n9NLw17+2FqmZWdfQOJ3FGWd0ftV5KSWFAcCUiJgWEXOA0cDehQdExPSIeB5YoEVY0hbAysD9Bcd/\nEhF/z36eCzwNrNZSAMOHw29/W0KkZmZdwC23pLEJBx3U+dcuJSn0BmYUvG7ItrVKkoBzgROAoq3g\nknoCg4CxLZ1n//3hlVfg6adLuaqZWfX67DM4+WQ45xzolkOrbymXLHYzL7WP6JHA3RHxr2LnkrQY\ncANwfkRMbekk3bvDT3/q0oKZdX2XX57GI+y2Wz7XL6VLagOwRsHr1YA3Sjz/NqT2giOBHkB3SR9F\nxCnZ/suBlyPiooWdZOTIkcyenYpUY8bUsf/+dSVe3sysenzwQWpHuP/+1o9trr6+nvr6+nbH0Org\ntexp/mVgF+BN4ElgSEQs0HNW0ijgroi4uci+ocAWEXFM9voMYP2I2K+V6zcNXjvuOFhssVSsMjPr\nan7xC3jjDRg1qv3nKuuIZkkDgQtI1U1XRsRZkk4DxkXEXZL6A7cCPYHZwFsRsXGzczQlBUmN7RQv\nAp+RqqMujoirily7KSk0rkv6+uuw3HKL+lHNzCpXQwNsuik8+yys1mK3m9LVzDQXBx4IAwakUoOZ\nWVfxwx/CKqvAmQsdtVW6mkkK48bB4MHw6qtpPhAzs2r33HOpYXnyZFh++Y45Z81Mnb3lltCnD9x0\nU96RmJl1jBNPhFNP7biE0B5VlxQgDWY799w06s/MrJo98ECq+Tj88LwjSaoyKey1F3z0ETz8cN6R\nmJm13fz5qZRw1lmwxBJ5R5NUZVLo1g1+9jMPZjOz6nb99Wluo30WmEY0P1XX0Nxo1qzUtvCPf8D6\n63d+XGZm7fHJJ+ne9ec/w7bbdvz5a6ahudEyy8ARR6S1S83Mqs2FF0L//uVJCO1RtSUFSGuXrr9+\n6sa10kqdHJiZWRv95z/Qt29aM2G99cpzjZorKQCsvHIas3DJJXlHYmZWujPOgAMOKF9CaI+qLilA\nWru0rg6mTvU6zmZW+V59FbbaCl54IT3YlktNlhQgFcH69/c6zmZWHU45JfWeLGdCaI+qLykAPPgg\nHHUUTJqUz6IUZmaleOIJ2Hff1A66zDLlvVbNlhQAdtopVR3dc0/ekZiZFde47vKvflX+hNAeXSIp\nSF7H2cwq2x13wPvvw9CheUeycF2i+ghgzhxYay24/fa05oKZWaWYMwc23jiNq9pjj865Zk1XH0Fa\nx/mYY1xaMLPKc+WVaeGcgQPzjqR1XaakADBzZiotPPssrL56mQMzMyvBRx+l8Qh33925tRg1X1IA\n6NkThg1Lw8fNzCrBuefCrrtWT7V2lyopgNdxNrPK8cYbqS3h6adhzTU799ouKWTWXDMta3fFFXlH\nYma1bsQIOOSQzk8I7dHlSgrgdZzNLH+TJqUxVJMnp6rtzuaSQoEtt0yZ2es4m1leTjoJfv7zfBJC\ne3TJpABw/PGpe2qFF4TMrAt66KE04d2RR+YdyaLrsklhr73gww/TymxmZp1l/vw0ncWvfw1LLpl3\nNIuuyyYFr+NsZnkYPTrdf/bfP+9I2qZLNjQ38jrOZtaZPv0UNtgArr4adtwx31jc0FzEMsvAj3/s\ndZzNrHNcfHEal5B3QmiPLl1SAHj77ZS5vY6zmZXTe++lGomHH06Lf+XNJYUWfOUraczCpZfmHYmZ\ndWVnnpkW0KmEhNAeJSUFSQMlvSRpsqSTiuzfXtJTkuZI2qfI/h6SGiRdWLDtDEnTJX3Yvo/QuuOO\ng0sugdmzy30lM6tFr78Oo0bByJF5R9J+rSYFSd2Ai4HdgY2AIZI2aHbYNGAo8KcWTnM6UN9s2x3A\nlosSbFv17QtbbAHXXdcZVzOzWnPqqWnq/l698o6k/UopKQwApkTEtIiYA4wG9i48ICKmR8TzwAKV\n/5K2AFYG7m/2nicj4u02R76Ihg+H885LfYjNzDrK+PFpsNrw4XlH0jFKSQq9gRkFrxuyba2SJOBc\n4ARgkRs8OtJOO8FSS3kdZzPrOI3rLo8cCcsum3c0HaOU6eKK3cxL7Q50JHB3RPwr5Ye2JYaRBRV1\ndXV11NXVLfI5pM+nvvjmN9sShZnZ5z78MLVXvvceHHxw3tFAfX099fX17T5Pq11SJW0NjIyIgdnr\nk4GIiLOLHDsKuDMibsleXw9sB8wHegDdgUsi4pSC93wYES2ufNDeLqmFvI6zmXWEBx6AQw9Ny2ue\nc05lrt3S1i6ppZQUxgHrSFoTeBM4EBiysFgaf4iIgwoCHApsUZgQmh9fboXrOP+ppSZxM7MWfPRR\nqnG491744x/hG9/IO6KO12qbQkTMA44mNRRPAkZHxIuSTpO0F4Ck/pJmAIOByyRNbO28ks7O3rN0\n1jX1f9v1SUr0ox+ldoUZM1o/1sys0dixabTy/Pnw3HNdMyFADYxoLuZnP0uL75xzToee1sy6oI8/\nhhNPhLvugssvT1VG1cAjmhfBT38KV12VGorMzFry0EOwySZp4Otzz1VPQmiPmkwKffqkdZyvvDLv\nSMysEn38MRx9NHz/+2mSu6uuqr4V1NqqJpMCpIEm558Pc+fmHYmZVZK//x023TQ1Kk+cCHvumXdE\nnatmk4LXcTazQv/9b6pa/t730gPjNdfACivkHVXnq9mkAKm04HWczewf/0ilg/feS20HgwblHVF+\najopDBoEH3zgdZzNatWsWak34gEHpAfE666DL30p76jyVdNJoVu3NEzd6zib1Z7HHoPNNoN33klt\nB3vv3fp7akFNjlMo5HWczWrLJ5/AL38JN9wAv/89fOc7eUdUHh6n0EZex9msdjz+OPTrBw0Nqe2g\nqyaE9qj5kgJ4HWezrm72bBgxAq69Fi66KC3R29W5pNAOX/lKWlvV6zibdT1PPplmRX7tNXj22dpI\nCO3hkkLmhRdg551h6tS0GI+ZVbdPP02L34waBRdeCPvvn3dEncslhXbacMO0jvP11+cdiZm11/jx\n6f/zyy+n0kGtJYT2cEmhwIMPpvlOnn8+dVc1s+ry6adw+ulprYPzz4cDD0yrLtYilxQ6wE47wZJL\neh1ns2r09NPQv396qHv2WRgypHYTQns4KRSQPp/6wsyqw2efpZ5Fe+wBJ50Et94KvXrlHVX1clJo\n5oADUtfUCRPyjsTMWvPMMzBgQColTJgABx3k0kF7OSk00717minRpQWzyjVnDpx2WloS87jj4I47\nYNVV846qa3BDcxEzZ8Jaa6V6ydVX79RLm1krnnsOhg2DVVZJy2P27p13RJXJDc0dqGdPGDo09W02\ns8owZ07qWbTrrvCTn6Q1k50QOp5LCi2YOjX1c379dVhuuU6/vJkVeP759KC20kqpu6lL8K1zSaGD\n9emTnki8jrNZfubOhTPPTN3FjzgidRd3QigvlxQWYtw42G8/eOUVWHzxXEIwq1kvvJBKBz17poez\nNdbIO6Lq4pJCGWy5ZfqHePPNeUdiVjvmzoWzz4Ydd4Qf/Qjuv98JoTO5pNCK22+HM85IMy26/7NZ\neb34YupZtOyyqXTQp0/eEVUvlxTKxOs4m5XfvHlwzjmwww4pKTzwgBNCXlxT3opu3dLC3r/9bfoH\na2Yd6+WX4Yc/TPOOPfkkfPWreUdU21xSKMHQofDPf6bpL8ysY8ybB+edB9tuC9/9Lowd64RQCUpK\nCpIGSnpJ0mRJJxXZv72kpyTNkbRPkf09JDVIurBg2+aSnsvOeX77PkZ5eR1ns441ZUpqSL7tNnji\niTRlvaerrwyt/hokdQMuBnYHNgKGSNqg2WHTgKHAn1o4zelAfbNtlwKHRsR6wHqSdl+EuDvdUUfB\n6NHw73/nHYlZ9Zo/Hy64ALbZJi18U18Pa6+dd1RWqJTcPACYEhHTImIOMBrYu/CAiJgeEc8DC3QT\nkrQFsDJwf8G2XkCPiHgy23Qt8O22fYTO4XWczdrnlVegrg5uvDFVxx5zjEsHlaiUX0lvYEbB64Zs\nW6skCTgXOAEo7BrVOzvPIp8zT8cdB5dcArNn5x2JWfWYPx8uugi23hq+8x34+99h3XXzjspaUkrv\no2L9XEsdOHAkcHdE/Etf7OS/SOccOXJk0891dXXU1dWVePmOteGGsPnmaR3nQw/NJQSzqvLaa3Dw\nwWkhnEcfhfXXzzuirqu+vp76+vp2n6fVwWuStgZGRsTA7PXJQETE2UWOHQXcGRG3ZK+vB7YD5gM9\ngO7AJcCFwEMR0Tc77kBgx4g4osg5cx281pzXcTZr3fz5cNllaUW0k0+GY4+FxRbLO6ra0tbBa6WU\nFMYB60haE3gTOBAYsrBYGn+IiIMKAhwKbBERp2SvP5Q0IDv/D0iJouI1ruN8772w5555R2NWeaZO\nhUMOgf/+Nw363KB5txSraK0+60bEPOBoUkPxJGB0RLwo6TRJewFI6i9pBjAYuEzSxBKufSRwJTCZ\n1JB9b1s/RGdqXMf53HPzjsSsskTAH/6Q5gzbffdUXeSEUH0891EbzJmTBtnceSf065d3NGb5mzYt\ntbN98AFcfXVqf7N8ee6jTtS9e+pO53WcrdZFpEVv+veHXXaBxx5zQqh2Lim0UeM6zs89B6utlnc0\nZp1vxow0tfV//pNKB1/7Wt4RWSGXFDqZ13G2WhWRprXefHPYfvs0EM0JoetwSaEdvI6z1ZqGhlQ6\nePvtVDrYZJO8I7KWuKSQA6/jbLUiIiWBzTdP8xY98YQTQlflkkI7PflkmtjL6zhbV/XGG3DYYamU\ncPXVsNlmeUdkpXBJIScDBsDqq3sdZ+t6IuC661IS6N8/PQA5IXR9Lil0gNtvh//7v1Sk9jrO1hW8\n9RYcfnhqL2usNrLq4pJCjgYNSl1UH3kk70jM2u8f/0gdKDbeGMaPd0KoNU4KHaBxHWdPfWHVLAIu\nvhgGD06dJ844A5ZYIu+orLO5+qiDzJqVeiM98gist17e0Zgtmk8+SUvOTpgAt97q1dC6Alcf5WyZ\nZVIdrNdxtmozbRpst11a8+Cf/3RCqHUuKXSgt99Os0JOmQIrrph3NGate/BB+O534cQTUxWoO0p0\nHS4pVACv42zVIiJN6Pi978ENN6SlZp0QDFxS6HAvvAA775ymwFhqqbyjMVvQf/+bprmeMgVuuQXW\nWCPviKwcXFKoEIXrOJtVmldfTdNULLlk6nrqhGDNOSmUwfDhcN55aZ1as0px773w9a+nDhGjRsHS\nS+cdkVUiJ4Uy2Hnn1L/73qpYYNS6ugg488y0bvJNN8FRR7n9wFrmNoUyuf769DQ2dmzekVgt+/BD\nGDYM3nwzJYTevfOOyDqL2xQqzAEHwOTJaTCQWR5efhm22gpWXhnq650QrDROCmXSvTv85Cdex9ny\ncfvtaVW04cPhsstSw7JZKVx9VEZex9k62/z5MHJkqrq86aZUUrDa1NbqIyeFMjv22NTo/Jvf5B2J\ndXUzZ6bBaB99BDfemAZTWu1ym0KFOvZYuOqq9B/VrFyefx623BLWWSd1bnBCsLZyUiizPn1gl128\njrOVz403wk47wf/+L1xwQWrPMmsrVx91Aq/jbOUwdy784hcwZkxaDtaL4VghVx9VMK/jbB3t3Xdh\njz3gqadg3DgnBOs4TgqdZPjw1D21ygs9VgEmTID+/aFfvzRq3tO0W0cqKSlIGijpJUmTJZ1UZP/2\nkp6SNEfSPgXb15A0XtLTkiZKOrxg3wGSns22n9UxH6dyDRoE77/vdZytfa6/Hr7xDTjrrNSjzdWR\n1tFabVOQ1A2YDOwCvAGMAw6MiJcKjlkDWA44HrgjIm7Jti+eXWOOpGWAScA2wGfABKBfRLwnaRRw\nbUQ8VOT6Vd+m0OjSS+G+++C22/KOxKrNnDlw/PFw991pucyNN847Iqt05WxTGABMiYhpETEHGA3s\nXXhAREyPiOeBaLZ9bvYegKWBxgDXAl6OiPey12OBfRc1+GozdCg89lia/sKsVG+/DbvumtY/GDfO\nCcHKq5Sk0BuYUfC6IdtWEkmrSXoWmAacHRFvAa8AG2TVS4sD3wZWLz3s6uR1nG1RPflkGn+www5w\n552wwgp5R2RdXSk1ksWKHyXX50REA7CppF7A7ZJuioh/SzoCGAPMAx4jlR6KGjlyZNPPdXV11NXV\nlXr5inPUUdC3L5x+uhsIbeGuvBJ+/nO4/HL49rfzjsYqXX19PfX19e0+TyltClsDIyNiYPb6ZCAi\n4uwix44C7mxsUyiy/yrgrub7Jf0IWDsiTi7yni7TptDokEPSoLZf/jLvSKwSffop/PSnaWbT226D\nDTbIOyKrRuVsUxgHrCNpTUlLAAcCdywsloKgektaKvt5BWBb4OXs9UoF248ErljU4KvVccfB738P\ns2fnHYlVmjfeSKOT3347VR05IVhnazUpRMQ84GjgflLvodER8aKk0yTtBSCpv6QZwGDgMkkTs7f3\nBZ6QNAF4CPhNREzK9l0gaRLwD+DMiHilQz9ZBdtoozTY6E9/yjsSqySPPJLaD/bcMw10XG65vCOy\nWuRpLnIydmxab+H556GbhxDWtIjUXXnkSLjmmjRS2ay92lp95KEvOSlcx3nPPfOOxvIyezYccQSM\nH5+6K6+zTt4RWa3zM2pOpM+nvrDaNH16Wh1t1iz45z+dEKwyOCnk6IAD0jq6zzyTdyTW2R56KK2K\ndsABMHo0LLts3hGZJU4KOVpiCTjmGJcWakkEnHceDBkC112Xpq7QItf6mpWPG5pz5nWca8esWXDo\nofDSS3DLLWmsilm5eD2FKtWzJ/zgB3DhhXlHYuX02muwzTZpVtNHH3VCsMrlkkIFeP31ND/+1KnQ\no0fe0VhHu+++lPhPPRWOPtrVRdY5XFKoYl/9qtdx7ooi4Ne/hh/+MK2j/JOfOCFY5XNJoUI88UTq\nieJ1nLuGjz6CYcOgoSGNTnZ7kXU2lxSq3FZbpXWcbyk6laBVk5dfTr/PL30JHn7YCcGqi5NCBRk+\nHM491+s4V7M770wD0o49Fv74R1hyybwjMls0TgoVxOs4V6/589PcRUccAbffDocdlndEZm3j2usK\nsthi8LOfpcFs22+fdzRWqpkz4fvfT3+PHw+9euUdkVnbuaRQYYYNSxOjTZmSdyRWikmTYMCANO5g\n7FgnBKt+TgoVxus4V4+bboK6OvjFL+Cii9K0JWbVzl1SK9Bbb6V1nKdM8TrOlWjevJQIRo9O3U23\n2CLviMwW5C6pXUivXrDPPmnhFass776b1r8YNy79cUKwrsZJoUJ5HefK88wzabnMjTdOU1estFLe\nEZl1PCeFCrXRRtCvn9dxrhQ33AC77QZnnpnGknjUuXVVblOoYF7HOX9z5sCJJ8Idd8Ctt8Imm+Qd\nkVlp3KZ/DFjNAAAK1UlEQVTQBRWu42yd7513UungxRdT+4ETgtUCJ4UK5nWc8zNuXGo/2HZbuPvu\nNI+RWS1w9VGF++yztDLbXXfBZpvlHU1tuOoqOOkk+MMfUi8ws2rU1uojJ4UqcPbZqV3huuvyjqRr\n++yzNJHd2LGp/WDDDfOOyKztnBS6MK/jXH5vvgmDB6fBgtdeC8svn3dEZu3jhuYurHEd54suyjuS\nrumxx1L7wcCBqYTghGC1zCWFKuF1nDteBFx2GYwYAaNGwTe/mXdEZh3HJYUuzus4d6zZs+HQQ9Oo\n8UcfdUIwa1RSUpA0UNJLkiZLOqnI/u0lPSVpjqR9CravIWm8pKclTZR0eMG+IZKek/SMpL9Kcqe/\nVgwfDuefD3Pn5h1JdZsxA3bYAT78EB5/HNZdN++IzCpHq0lBUjfgYmB3YCNgiKQNmh02DRgKNJ+U\n4Q1gm4jYHNgKOFlSL0mLAecDO0bEZsBE4Oh2fZIa4HWc2ycCHnwwrX8weDCMGQPLLpt3VGaVpZQZ\nXAYAUyJiGoCk0cDewEuNB0TE9GzfFyr/I6LwmXZpoLF+q/HvHpJmAssBXlamBMOHp/l39tsvDW6z\nBb33HkyenKYeb/53z56pd9Fuu+UdpVllKiUp9AZmFLxuICWKkkhaDbgbWBs4ISLeyrYfSSohfExK\nCEeWes5aNmgQnHBCWse5lpfs/Pjj4jf9yZNT9dp666VqofXWS99Z4+uePfOO3KyylZIUij2Pltwd\nKCIagE0l9QJul3QT8D5wBLBpREyVdBFwCvB/pZ63VtXSOs6zZ8Orrxa/+c+cCeus8/nNfqed4LDD\n0uuVVnIpyqytSkkKDcAaBa9XI7UVLJKIeEvSJGB7YHraFFOz3WOABRqwG40cObLp57q6Ourq6hb1\n8l3KsGGpG+WUKdXfSDp3bupmW+yJ/6230trHjU/8W2wBQ4ak1717e+ZYs0L19fXU19e3+zytjlPI\nGoVfBnYB3gSeBIZExItFjh0F3BURN2evewPvRsRsSSsAjwP7AO8B44FNIuJdSb8Clo6IE4qc0+MU\nijj11FR3fskleUfSuvnzoaGh+I1/+nRYZZUvVvc0/r3mml63wKytyjrNhaSBwAWk3kpXRsRZkk4D\nxkXEXZL6A7cCPYHZwFsRsbGkXYHfAvNJ1VAXRcSV2TkPA44FPiP1XhoWEe8XubaTQhFvvZXm5pky\nBb785byjST173n67eFXPq6/CCisUv/GvtRYsuWTe0Zt1PZ77qAYdckga1HbqqZ13zfffb7lnz5JL\nLnjTX3fdVPfvrp9mnctJoQZNmpRGOU+dCkst1XHn/fhjeOWV4jf/zz4r/sS/7rqpNGBmlcFJoUbt\nsUcaiHXIIYv2vk8/bblnz/vvw9prF7/5r7yye/aYVQMnhRr1t7/BT3+a1ltofrOeOxemTSv+xP/m\nm6kht9gT/2qruWePWbVzUqhREdCvX5pae5llvnjznzoVevUq/sTfp4979ph1ZU4KNezOO9NgtsIb\n/7rrpiqgjmxrMLPq4aRgZmZNvJ6CmZm1m5OCmZk1cVIwM7MmTgpmZtbEScHMzJo4KZiZWRMnBTMz\na+KkYGZmTZwUzMysiZOCmZk1cVIwM7MmTgpmZtbEScHMzJo4KZiZWRMnBTMza+KkYGZmTZwUzMys\niZOCmZk1cVIwM7MmTgpmZtbEScHMzJo4KZiZWZOSkoKkgZJekjRZ0klF9m8v6SlJcyTtU7B9DUnj\nJT0taaKkw7Pty0qakG2fIOnfks7ruI9lZmZt0WpSkNQNuBjYHdgIGCJpg2aHTQOGAn9qtv0NYJuI\n2BzYCjhZUq+I+Dgi+kXE5hHRL3v/ze38LJ2mvr4+7xAW4JhKV4lxOabSOKbyK6WkMACYEhHTImIO\nMBrYu/CAiJgeEc8D0Wz73Ow9AEsDan5ySesCK0XEo235AHmoxH8Ejql0lRiXYyqNYyq/UpJCb2BG\nweuGbFtJJK0m6VlSaeDsiHir2SEHAn8p9XxmZlY+pSSFBZ7uaVYiWJiIaIiITYF1gGGSVmp2yIHA\nn0s9n5mZlY8iFn5/l7Q1MDIiBmavTwYiIs4ucuwo4M6IuKWFc10F3NW4X9ImwJiIaN5GUfiekhOQ\nmZl9LiKKPdQv1OIlHDMOWEfSmsCbpCf7IQs5vikISb2BdyNitqQVgG2B3xYcO4RWSglt+VBmZtY2\nrVYfRcQ84GjgfmASMDoiXpR0mqS9ACT1lzQDGAxcJmli9va+wBOSJgAPAb+JiEkFp98PVx2ZmVWM\nVquPzMysdlTEiOa2Do6rgLh+JmmSpGckPSBp9QqI6XBJz2WDAh8uMqak02MqOG6wpPmSNs87JklD\nJb2TDaB8WtLBeceUHbN/9m9qoqTryx1TKXFJOq9gsOnLkt6rgJhWl/RgFtMzkvaogJjWkPQ3Sc9m\nsa3aCTFdKeltSc8t5JgLJU3JvqfNWj1pROT6h5SYXgHWBLoDzwAbNDtmDeBrwNXAPhUU147AUtnP\nPyZVreUd07IFPw8C7sk7psa4gL8DjwGb5x0TabDlhZ3xb2kRYloHeApYLnu9YiXE1ez4o4Er8o4J\n+ANwePZzX+D1CohpDHBQ9nMdcG0n/P62AzYDnmth/x7A3dnPWwGPt3bOSigptHlwXAXE9feImJ29\nfJxFGL9Rxpg+Lni5LDA/75gypwNnA5+WOZ5FiakzOzGUEtOPgN9HxIcAEfGfComrUKudQzoppvnA\nctnPPYF/VUBMGwIPAkREfZH9HS4iHgHeX8ghewPXZsc+ASwv6SsLO2clJIV2DY4ro0WN6xDgnrJG\nVGJMko6U9ApwFnBM3jFlRdbVIuKvZY6l5Jgy+2RF6jGSVquAmNYD1pf0iKTHJO1e5phKjQtI1SNA\nH7IbX84xnQZ8P+vgchfwkwqI6RlgX4CsmnvZrNdlnprH/S9aub9WQlJo1+C4Mio5LkkHAVsA55Q1\nohJjiohLImId4CTgl3nGJEnA74Dhrbyn02LK3AH0iYjNgLHANRUQ0+KkKqQdgO8CV0haboF3dX5c\njQ4EboqsLqKMSolpCDAqIlYHvgmUu/2llJhOAOokPQVsT7oBzy1zXK1Z5PtrJSSFBlKbQaPVSBPp\n5a2kuCTtCvwcGBSfz/OUa0wF/gJ8u6wRtR5TD9JEivWSXge2Bm4vc2Nzq99TRLxf8Pv6Iympl1Mp\nv7sG4PaImB8RU4GXgXUrIK5GnTX7QCkxHUKqwyciHgeWkrRinjFFxJsRsW9EbAGcmm37qIwxlaIB\nKOwA0/r9tdwNISU0lCzG5w04S5CKYH1bOHYUsG+lxAX0y45Zu4JiWqfg50HAk3nH1Oz4h4B+eccE\n9Cr4+TvAYxUQ0+7A1dnPK5LmC1sh77iy49YHXitnLIv4Xd0NDM1+7gs0VEBMX+bzbv5nkGaC6Izv\nqw8wsYV9e/J5Q/PWlNDQXPaAS/xQA0lPRVOAk7NtpwF7ZT/3J9WLfQT8u6UvIIe4HiCN8n4amADc\nVgExnQ88n8U0dmE36M6KqdmxD1Lm3kclfk9nZt/ThOx7Wi/vmLLXvyUNEn0W2K/cMS1CXCOAMzsj\nnhJ/f32BR7Kb89PALhUQ077AZOAl4HKgeyfEdAPpyf9TYDrwQ+Bw4LCCYy4mJbRnS/m/58FrZmbW\npBLaFMzMrEI4KZiZWRMnBTMza+KkYGZmTZwUzMysiZOCmZk1cVIwM7MmTgpmZtbk/wFZP8Bd0neL\nOQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108f69ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(max_features, test_scores)\n",
    "plt.title(\"Max Features vs CV Error\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用RF的最优值达到了0.137"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Ensemble\n",
    "\n",
    "这里我们用一个Stacking的思维来汲取两种或者多种模型的优点\n",
    "\n",
    "首先，我们把最好的parameter拿出来，做成我们最终的model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ridge = Ridge(alpha=15)\n",
    "rf = RandomForestRegressor(n_estimators=500, max_features=.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n",
       "           max_features=0.3, max_leaf_nodes=None, min_impurity_split=1e-07,\n",
       "           min_samples_leaf=1, min_samples_split=2,\n",
       "           min_weight_fraction_leaf=0.0, n_estimators=500, n_jobs=1,\n",
       "           oob_score=False, random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge.fit(X_train, y_train)\n",
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面提到了，因为最前面我们给label做了个log(1+x), 于是这里我们需要把predit的值给exp回去，并且减掉那个\"1\"\n",
    "\n",
    "所以就是我们的expm1()函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_ridge = np.expm1(ridge.predict(X_test))\n",
    "y_rf = np.expm1(rf.predict(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个正经的Ensemble是把这群model的预测结果作为新的input，再做一次预测。这里我们简单的方法，就是直接『平均化』。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_final = (y_ridge + y_rf) / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: 提交结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "submission_df = pd.DataFrame(data= {'Id' : test_df.index, 'SalePrice': y_final})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们的submission大概长这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1461</td>\n",
       "      <td>119595.627405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1462</td>\n",
       "      <td>152127.359971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1463</td>\n",
       "      <td>174472.484621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1464</td>\n",
       "      <td>189936.942219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1465</td>\n",
       "      <td>193934.290197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1466</td>\n",
       "      <td>175889.222850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1467</td>\n",
       "      <td>177835.726832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1468</td>\n",
       "      <td>169239.114752</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1469</td>\n",
       "      <td>184864.220939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1470</td>\n",
       "      <td>123773.699896</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Id      SalePrice\n",
       "0  1461  119595.627405\n",
       "1  1462  152127.359971\n",
       "2  1463  174472.484621\n",
       "3  1464  189936.942219\n",
       "4  1465  193934.290197\n",
       "5  1466  175889.222850\n",
       "6  1467  177835.726832\n",
       "7  1468  169239.114752\n",
       "8  1469  184864.220939\n",
       "9  1470  123773.699896"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "走你~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
